Hello team, Are there any plans to support multi ...
# support
s
Hello team, Are there any plans to support multi clickhouse cluster from the query-service? A separate clickhouse cluster for logs, metrics and traces?
s
What's the use case?
s
For our use case we have TBs of metrics and logs flowing in, we want to store logs and metrics in different Clickhouse clusters and then query it using query-service
s
Why do you want to store them separately?
s
Won't a single Clickhouse cluster be a single point of failure so much of logs and metrics data?
s
I am not sure how the per-signal cluster solves that. You will need to use replication if the concern is the single point of failure.
s
Let's say if there are a lot metrics being written to the Clickhouse cluster and the cluster gets overloaded. This can bottleneck the logs pipeline from sending logs to the clickhouse cluster
s
You would solve that by capacity planning, and putting query restrictions so that some rouge querying/ingestion won't cause problems to the entire cluster etc..
Roughly how much data do you ingest and retain over a period? few hundreds of terabytes don't warrant such a need for signal-specific cluster requirement.
s
Understood, thank you! yes the log and metrics data is in TBs. we keep it for close to 2 months