Just noticed in my custom clickhouse cluster that ...
# supporta
Alexei Zenin
08/03/2022, 10:55 PMJust noticed in my custom clickhouse cluster that the SigNoz tables are not created on every node, what is the recommended approach to get all the tables to align across all the Clickhouse nodes? I’m trying to get HA but have noticed most of the push is to run 1 instance of clickhouse atm. Does that scale well, is a cluster overkill?
Alexei Zenin
08/03/2022, 10:57 PMnever worked with Clickhouse but surprised they don’t replicate DDL: https://clickhouse.com/docs/en/engines/table-engines/mergetree-family/replication/
a
Ankit Nayan
08/04/2022, 4:21 AM
hey @Alexei Zenin we have not enabled replicated mergetree till now. Though it is planned in soon.
Ankit Nayan
08/04/2022, 4:22 AM
how many spans/s do you expect to write to clickhouse? We tested with 500K spans/s ingestion into single clickhouse instance which consumed around 10-15 CPUs
Ankit Nayan
08/04/2022, 4:23 AM
ReplicatedMergeTree is recommended even for that load to enable data redundancy and reduce single point of failure.
Ankit Nayan
08/04/2022, 4:24 AM
Let us know if you must move to ReplicatedMergeTree very soon, we might help with that too else it will come as planned within 2 months
a
Alexei Zenin
08/04/2022, 1:03 PMI see, it should be fine I think. Wasn't aware of the hard limitation that SigNoz can only use 1 clickhouse instance at the moment. Thanks for clarifying
Alexei Zenin
08/04/2022, 5:20 PMOnly concern is HA of our setup and if the node goes down. Concerning that all of our observabilty could disappear due to 1 machine. For PROD readiness I would definitely expect an HA setup.
15 Views