Specifically - we slowly increase the load of micr...
# support
r
Specifically - we slowly increase the load of micro-service data - but we are facing challenges - with the query builder and unable to load. - any idea or suggestion - or any pointer talks about the scalability of the system.
s
If you configured the tiered volume policy for the data clickhouse reads it from s3. How much load/scale are you planning for?
It takes orders of magnitude to complete operations with s3
r
Well, we wanted to scale for 200 micro-service
Then I got the information - OTEL collector and frontend query builder are not able to scale - i mean more no of replica in the yaml
Our dev-ops team member infact submitted code in the signoz.
s
Did you mean query-service? It loads the time series in memory to support prometheus. What is the volume of data ingested and retention requirements?
p
@rakesh shah Otel Collector and Frontend can be horizontally scaled as per your need.
Alertmanager, we haven't tested the cluster mode with multiple peers. @Amol Umbark should be able to answer it better.
OtelCollector Metrics is not required to be scaled, since it is used for pull-based scraping.
r
OK.
p
Query-Service is currently not scalable mainly because of dependency to SQLiteDB for storing users, orgs and dashboards data. When we migrate to PostgreSQL, Query-Service should be horizontally scalable.
r
OK
Queryservice - is talking to ClickHouse -
Does any thing cached for query service ?
s
Yes, the time series is cached for prometheus support. What is the approximate number of time series you generate?
r
We don't use prometheus - until it is used by Signoz internally
s
What metric system do you use? Or you are just using signoz just for tracing?
r
How metric data is displayed by signoz?
Even we provide only tracing details, signoz is showing metric? - How is it computed by signoz?
s
signoz is showing metric? - How is it computed by signoz?
They are generated from spans data.
r
OK.
a
@rakesh shah is signoz working smoothly or you still facing problems in query-builder?
r
Hi Ankit, We have moved time-being to grafana tempp
a
ok. no worries. check us out sometime later when you might need some aggregations and custom dashboards on trace data. BTW was the loading issue of query-service primarily the reason?