Hi all! i'm having trouble with signoz deployment (helm chart 0.21.0) otel collector show messages suc has

2023-07-31T10:55:05.548Z        error   exporterhelper/queued_retry.go:391      Exporting failed. The error is not retryable. Dropping data.     {"kind": "exporter", "data_type": "metrics", "name": "clickhousemetricswrite", "error": "Permanent error: unsupported metric type; Permanent error: unsupported metric type; Permanent error: unsupported metric type; Permanent error: unsupported metric type", "errorCauses": [{"error": "Permanent error: unsupported metric type"}, {"error": "Permanent error: unsupported metric type"}, {"error": "Permanent error: unsupported metric type"}, {"error": "Permanent error: unsupported metric type"}], "dropped_items": 6738}
<http://go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send>
        /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.79.0/exporterhelper/queued_retry.go:391
<http://go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send>
        /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.79.0/exporterhelper/metrics.go:125
<http://go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1|go.opentelemetry.io/collector/exporter/exporterhelper.(*queuedRetrySender).start.func1>
        /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.79.0/exporterhelper/queued_retry.go:195
<http://go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func1|go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).StartConsumers.func1>
        /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.79.0/exporterhelper/internal/bounded_memory_queue.go:47

agent logs

2023-07-31T10:54:12.677Z        info    exporterhelper/queued_retry.go:423      Exporting failed. Will retry the request after interval.        {"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 1signoz-otel-collector:4317: connect: connection refused\"", "interval": "10.268880523s"}
2023-07-31T10:54:22.948Z        info    exporterhelper/queued_retry.go:423      Exporting failed. Will retry the request after interval.        {"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp signoz-otel-collector:4317: connect: connection refused\"", "interval": "37.016329507s"}

clickhouse memory usage is above 30Gb the cluster is rather stale and has no much going on there, yet it looks like the whole stack is overflowed any ideas?

You may be running into this same error https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1690567504159129

interesting, thank you, I'll follow the thread on GitHub.