Hi Team, We are looking to self-host Signoz on a dedicated cluster, independent from application k8s cluster which ensures access to our monitoring data during application cluster outages and helps us in troubleshooting. For this, we are trying to setup OTEL collectors on the application collector to collect metrics, traces & logs and send them directly to Signoz's clickhouse database directly. But the default Signoz helm chart supports only receiving data via OTEL collectors only which would waste our resources in running OTEL collectors at scale on both our application & monitoring cluster. Is there any way we can configure the monitoring cluster's Clickhouse receiver in the application cluster OTEL collector?

We are doing the same thing. How we set it up: • SigNoz cluster with an ingest DNS routing traffic to the signoz collector ◦ This takes OTEL data and transforms it into SigNoz ClickHouse to all your data is saves and can be plotted on things like dashboards, alerted on, etc. • Our other clusters "ship" OTEL data to this collector. We just set the host to that ingress DNS from above. This is done in 3 different flavors: ◦ k8s-signoz chart installed collector to ship k8s metrics for the cluster (node counts, pod states, pod logs, etc.) ◦ annotated pods to "inject" a sidecar that ships APM data ◦ OTEL SDK baked into our apps that ship APM data as well as custom metrics

In this way the same collectors are running both on the application cluster and the signoz cluster. Instead if we can just host the Clickhouse receiver on the monitoring cluster and Signoz OTEL collector on the application cluster would solve the problem I was referring. Is there any way where I can just run the collectors on the application cluster and other services on the monitoring cluster?

It won't be a good idea. The collectors near the application have some diff in config than near the clickhouse. Unless you plan to run on small scale, this is going to create problems