https://signoz.io logo
Title
r

rakesh shah

08/11/2022, 6:42 AM
Specifically - we slowly increase the load of micro-service data - but we are facing challenges - with the query builder and unable to load. - any idea or suggestion - or any pointer talks about the scalability of the system.
s

Srikanth Chekuri

08/11/2022, 6:46 AM
If you configured the tiered volume policy for the data clickhouse reads it from s3. How much load/scale are you planning for?
It takes orders of magnitude to complete operations with s3
r

rakesh shah

08/11/2022, 6:48 AM
Well, we wanted to scale for 200 micro-service
Then I got the information - OTEL collector and frontend query builder are not able to scale - i mean more no of replica in the yaml
Our dev-ops team member infact submitted code in the signoz.
s

Srikanth Chekuri

08/11/2022, 6:52 AM
Did you mean query-service? It loads the time series in memory to support prometheus. What is the volume of data ingested and retention requirements?
p

Prashant Shahi

08/11/2022, 6:59 AM
@rakesh shah Otel Collector and Frontend can be horizontally scaled as per your need.
Alertmanager, we haven't tested the cluster mode with multiple peers. @Amol Umbark should be able to answer it better.
OtelCollector Metrics is not required to be scaled, since it is used for pull-based scraping.
r

rakesh shah

08/11/2022, 7:02 AM
OK.
p

Prashant Shahi

08/11/2022, 7:03 AM
Query-Service is currently not scalable mainly because of dependency to SQLiteDB for storing users, orgs and dashboards data. When we migrate to PostgreSQL, Query-Service should be horizontally scalable.
r

rakesh shah

08/11/2022, 7:04 AM
OK
Queryservice - is talking to ClickHouse -
Does any thing cached for query service ?
s

Srikanth Chekuri

08/11/2022, 7:06 AM
Yes, the time series is cached for prometheus support. What is the approximate number of time series you generate?
r

rakesh shah

08/11/2022, 7:12 AM
We don't use prometheus - until it is used by Signoz internally
s

Srikanth Chekuri

08/11/2022, 7:16 AM
What metric system do you use? Or you are just using signoz just for tracing?
r

rakesh shah

08/11/2022, 7:20 AM
How metric data is displayed by signoz?
Even we provide only tracing details, signoz is showing metric? - How is it computed by signoz?
s

Srikanth Chekuri

08/11/2022, 7:21 AM
signoz is showing metric? - How is it computed by signoz?
They are generated from spans data.
r

rakesh shah

08/11/2022, 1:46 PM
OK.
a

Ankit Nayan

08/22/2022, 11:35 AM
@rakesh shah is signoz working smoothly or you still facing problems in query-builder?
r

rakesh shah

08/22/2022, 4:31 PM
Hi Ankit, We have moved time-being to grafana tempp
a

Ankit Nayan

08/23/2022, 4:15 AM
ok. no worries. check us out sometime later when you might need some aggregations and custom dashboards on trace data. BTW was the loading issue of query-service primarily the reason?