hello, is there a way to get a total execution time of a signoz dashboard? My dashboard has 4-5 graphs and I would like to know how long it takes to generate the whole dashboard. Also is there is a way to get each separate graph/table total execution time (I guess I can try to get this from the clickhouse database, but it's just the time needed for the database to exctract the data, not counting time needed from other signoz elemets to generate graph/table/etc) (aside from maybe exporting each graph/table into separate dashboards if there is only a way to get a dashboard time, not separate graphs/tables) thx!

Hi @WD trying to understand the use case here. what is the use case related to calculating the time to load a dashboard in signoz ?

We are benchmarking/testing performance with/without using clickhouse shards, trying different configuration options, trying limiting resources in k8s and we want to see how it actually affects the execution time for dashboards 🙂

If the experimentation is more or less around clickhouse then the major time that you would be concerned about is the query-response time. the UI would be a constant factor here. But if you are still interested in checking the UI load times , lighthouse report / web-vitals and performance benchmarking in the chrome dev tools should be able to help you out.

😞 is there a dashboard for signoz we could use to display those query times inside signoz (without having to dig into clikchouse or use browser dev tools) or maybe a way to extract the queries generated by signoz to run them directly against clickhouse?

I am assuming finding the queries shouldn't be hard if you check the query-service logs / ch logs . @Srikanth Chekuri is there any easy way to retrieve this ?

Yeah, that doesn't work well when we have like 800k logs being send to signoz every minute and it's hard to find SELECT queries when there are almost only INSERT ones in logs 😉 (the amount of data send is the main reason we work on performance 😉 )

@WD, could you not monitor the clickhouse logs with something like fluentbit and add a filter to only consider the lines you are interested in, i.e. SELECT?

I thought that maybe signoz has some out-of-the-box solution for this as I think that adding fluentbit (or other log forwarder soft) to clickhouse in kubernetes setup might actually be way more complicated... As for the second suggestion - I would prefer to open a dashboard and notice "total time: xxx seconds" instead of having to sit with browser dev tools (which might not show me correct time) or use a pocket watch to measure the time until everything loads thanks for suggestions though @Daniel Hilgarth , when other options won't help I'll probably use one of yours 🙂