Hi SigNoz community! I am running on a k3s Kubernetes cluster, with SigNoz deployed via the SigNoz Helm chart. I can get logs flowing in OK with the

filelog

reciever. However I see no metrics coming in. I am pretty sure that I have configured the

hostmetrics

reciever fine as a Daemonset. Any insights on what I missed would be greatly appreciated. Details in this thread.

This message was deleted.

After upgrading to the last version, i noted a massive number of the same error being throw in the logs from the clickhouse container:

signoz-clickhouse      | 2024.04.30 15:33:58.073145 [ 623 ] {} <Error> MergeTreeBackgroundExecutor: Exception while executing background task {1d6b00a2-62a1-426a-a406-f72d7dd97975::20240430_1_122_8}: Code: 117. DB::Exception: Invalid flag for negative store: (while reading from part /var/lib/clickhouse/store/1d6/1d6b00a2-62a1-426a-a406-f72d7dd97975/20240430_1_36_7/ in table signoz_metrics.exp_hist (1d6b00a2-62a1-426a-a406-f72d7dd97975) located on disk default of type local, from mark 0 with max_rows_to_read = 8192): While executing MergeTreeSequentialSource. (INCORRECT_DATA), Stack trace (when copying this message, always include the lines below):
signoz-clickhouse      | 
signoz-clickhouse      | 0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c800f1b in /usr/bin/clickhouse
signoz-clickhouse      | 1. DB::Exception::Exception<>(int, FormatStringHelperImpl<>) @ 0x000000000721a243 in /usr/bin/clickhouse
signoz-clickhouse      | 2. DB::DDSketchDenseLogarithmic::deserialize(DB::ReadBuffer&) @ 0x000000000d5260e0 in /usr/bin/clickhouse
signoz-clickhouse      | 3. DB::SerializationAggregateFunction::deserializeBinaryBulk(DB::IColumn&, DB::ReadBuffer&, unsigned long, double) const @ 0x00000000108b58e9 in /usr/bin/clickhouse
signoz-clickhouse      | 4. DB::ISerialization::deserializeBinaryBulkWithMultipleStreams(COW<DB::IColumn>::immutable_ptr<DB::IColumn>&, unsigned long, DB::ISerialization::DeserializeBinaryBulkSettings&, std::shared_ptr<DB::ISerialization::DeserializeBinaryBulkState>&, std::unordered_map<String, COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::hash<String>, std::equal_to<String>, std::allocator<std::pair<String const, COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>>*) const @ 0x00000000108b01d9 in /usr/bin/clickhouse
signoz-clickhouse      | 5. DB::MergeTreeReaderCompact::readRows(unsigned long, unsigned long, bool, unsigned long, std::vector<COW<DB::IColumn>::immutable_ptr<DB::IColumn>, std::allocator<COW<DB::IColumn>::immutable_ptr<DB::IColumn>>>&) @ 0x000000001250a6b9 in /usr/bin/clickhouse
signoz-clickhouse      | 6. DB::MergeTreeSequentialSource::generate() @ 0x000000001251ca4d in /usr/bin/clickhouse
signoz-clickhouse      | 7. DB::ISource::tryGenerate() @ 0x000000001297acf5 in /usr/bin/clickhouse
signoz-clickhouse      | 8. DB::ISource::work() @ 0x000000001297a743 in /usr/bin/clickhouse
signoz-clickhouse      | 9. DB::ExecutionThreadContext::executeTask() @ 0x000000001299371a in /usr/bin/clickhouse
signoz-clickhouse      | 10. DB::PipelineExecutor::executeStepImpl(unsigned long, std::atomic<bool>*) @ 0x000000001298a170 in /usr/bin/clickhouse
signoz-clickhouse      | 11. DB::PipelineExecutor::executeStep(std::atomic<bool>*) @ 0x0000000012989928 in /usr/bin/clickhouse
signoz-clickhouse      | 12. DB::PullingPipelineExecutor::pull(DB::Chunk&) @ 0x0000000012998017 in /usr/bin/clickhouse
signoz-clickhouse      | 13. DB::PullingPipelineExecutor::pull(DB::Block&) @ 0x00000000129981d3 in /usr/bin/clickhouse
signoz-clickhouse      | 14. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::executeImpl() @ 0x000000001233b6f2 in /usr/bin/clickhouse
signoz-clickhouse      | 15. DB::MergeTask::ExecuteAndFinalizeHorizontalPart::execute() @ 0x000000001233b64b in /usr/bin/clickhouse
signoz-clickhouse      | 16. DB::MergeTask::execute() @ 0x0000000012340d99 in /usr/bin/clickhouse
signoz-clickhouse      | 17. DB::MergePlainMergeTreeTask::executeStep() @ 0x0000000012723517 in /usr/bin/clickhouse
signoz-clickhouse      | 18. DB::MergeTreeBackgroundExecutor<DB::DynamicRuntimeQueue>::threadFunction() @ 0x00000000123532c4 in /usr/bin/clickhouse
signoz-clickhouse      | 19. ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::worker(std::__list_iterator<ThreadFromGlobalPoolImpl<false>, void*>) @ 0x000000000c8eb0c1 in /usr/bin/clickhouse
signoz-clickhouse      | 20. void std::__function::__policy_invoker<void ()>::__call_impl<std::__function::__default_alloc_func<ThreadFromGlobalPoolImpl<false>::ThreadFromGlobalPoolImpl<void ThreadPoolImpl<ThreadFromGlobalPoolImpl<false>>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000000c8ee8fa in /usr/bin/clickhouse
signoz-clickhouse      | 21. void* std::__thread_proxy[abi:v15000]<std::tuple<std::unique_ptr<std::__thread_struct, std::default_delete<std::__thread_struct>>, void ThreadPoolImpl<std::thread>::scheduleImpl<void>(std::function<void ()>, Priority, std::optional<unsigned long>, bool)::'lambda0'()>>(void*) @ 0x000000000c8ed6fe in /usr/bin/clickhouse
signoz-clickhouse      | 22. ? @ 0x00007f8adb8b3609
signoz-clickhouse      | 23. ? @ 0x00007f8adb7d8353
signoz-clickhouse      |  (version 24.1.2.5 (official build))

This message was deleted.

This message was deleted.

hello everyone, I'm config otel using connectors routing to signoz exporter by the condition: attributes["service.name"]. Total have 12 pipelines so get error logs:

Error: failed to build pipelines: pipeline not found: missing consumer: "traces/default"; missing consumer: "metrics/default"

When reduce pipeline, it working fine. My question is max of pipelines which otel support ? • otel/opentelemetry-collector-contrib version 0.120.0

Hi all, Do Signoz has plans to implement the feature of session replay for frontend applications? https://docs.sentry.io/product/explore/session-replay/web/

Hi all, Is anyone using gunicorn with gevent worker class along with open telemetry auto instrumentation? We are using a django app with gunicorn Is it not yet compatible?

This message was deleted.

Hi all, I'm trying to deploy the

signoz-0.89.0

Helm chart to an EKS auto mode cluster. The deployment appears to succeed but the

signoz-0

pod keeps logging the error

{"level":"error","timestamp":"2025-08-22T05:34:16.103Z","caller":"opamp/opamp_server.go:117","msg":"Failed to find or create agent","agentID":"0198cff4-aeac-705d-90ef-b8db1ce6e534","error":"cannot create agent without orgId","errorVerbose":"cannot create agent without orgId\ngithub.com/SigNoz/signoz/pkg/query-service/app/opamp/model.(*Agents).FindOrCreateAgent\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/opamp/model/agents.go:91\ngithub.com/SigNoz/signoz/pkg/query-service/app/opamp.(*Server).OnMessage\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/opamp/opamp_server.go:115\ngithub.com/open-telemetry/opamp-go/server.(*server).handleWSConnection\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.19.0/server/serverimpl.go:253\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.23.11/x64/src/runtime/asm_amd64.s:1700","stacktrace":"github.com/SigNoz/signoz/pkg/query-service/app/opamp.(*Server).OnMessage\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/opamp/opamp_server.go:117\ngithub.com/open-telemetry/opamp-go/server.(*server).handleWSConnection\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.19.0/server/serverimpl.go:253"}

...and also on the

signoz-otel-collector

pod there are these errors (I assume related):

{"level":"error","timestamp":"2025-08-22T05:00:16.057Z","caller":"opamp/server_client.go:146","msg":"Server returned an error response: %v","component":"opamp-server-client","":"","stacktrace":"github.com/SigNoz/signoz-otel-collector/opamp.(*serverClient).Start.func3\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/server_client.go:146\ngithub.com/open-telemetry/opamp-go/client/internal.(*receivedProcessor).processErrorResponse\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.19.0/client/internal/receivedprocessor.go:247\ngithub.com/open-telemetry/opamp-go/client/internal.(*receivedProcessor).ProcessReceivedMessage\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.19.0/client/internal/receivedprocessor.go:170\ngithub.com/open-telemetry/opamp-go/client/internal.(*wsReceiver).ReceiverLoop\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.19.0/client/internal/wsreceiver.go:94"}

The helm chart deployed fine on a local k3s instance, I only see this issue on EKS. Any ideas how I can fix or investigate further? Thanks. (edit: corrected the first error message)