Hello everyone, I'm facing issue replate to Otel collector it's stuck in crashloopbackoff, when i checked the logs got the following.

{"level":"info","timestamp":"2025-01-08T05:19:57.460Z","caller":"signozcol/collector.go:168","msg":"Restarting collector service"}
{"level":"info","timestamp":"2025-01-08T05:19:57.460Z","caller":"signozcol/collector.go:144","msg":"Shutting down collector service"}
{"level":"info","timestamp":"2025-01-08T05:19:57.460Z","caller":"signozcol/collector.go:154","msg":"Collector service is not running"}
{"level":"info","timestamp":"2025-01-08T05:19:57.461Z","caller":"signozcol/collector.go:103","msg":"Starting collector service"}
{"level":"info","timestamp":"2025-01-08T05:19:57.470Z","caller":"signozcol/collector.go:168","msg":"Restarting collector service"}
{"level":"info","timestamp":"2025-01-08T05:19:57.470Z","caller":"signozcol/collector.go:144","msg":"Shutting down collector service"}
{"level":"info","timestamp":"2025-01-08T05:19:57.470Z","caller":"signozcol/collector.go:152","msg":"Collector service is shut down"}
{"level":"info","timestamp":"2025-01-08T05:19:57.471Z","caller":"signozcol/collector.go:103","msg":"Starting collector service"}
{"level":"error","timestamp":"2025-01-08T05:19:57.478Z","caller":"opamp/server_client.go:268","msg":"Collector failed for restart during rollback","component":"opamp-server-client","error":"failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:\n\n* error decoding 'exporters': error reading configuration for \"clickhousetraces\": 1 error(s) decoding:\n\n* '' has invalid keys: use_new_schema","stacktrace":"github.com/SigNoz/signoz-otel-collector/opamp.(*serverClient).reload\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/server_client.go:268\ngithub.com/SigNoz/signoz-otel-collector/opamp.(*agentConfigManager).applyRemoteConfig\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/config_manager.go:173\ngithub.com/SigNoz/signoz-otel-collector/opamp.(*agentConfigManager).Apply\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/config_manager.go:159\ngithub.com/SigNoz/signoz-otel-collector/opamp.(*serverClient).onRemoteConfigHandler\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/server_client.go:209\ngithub.com/SigNoz/signoz-otel-collector/opamp.(*serverClient).onMessageFuncHandler\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/server_client.go:199\ngithub.com/open-telemetry/opamp-go/client/types.CallbacksStruct.OnMessage\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/types/callbacks.go:162\ngithub.com/open-telemetry/opamp-go/client/internal.(*receivedProcessor).ProcessReceivedMessage\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/internal/receivedprocessor.go:131\ngithub.com/open-telemetry/opamp-go/client/internal.(*wsReceiver).ReceiverLoop\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/internal/wsreceiver.go:57\ngithub.com/open-telemetry/opamp-go/client.(*wsClient).runOneCycle\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/wsclient.go:243\ngithub.com/open-telemetry/opamp-go/client.(*wsClient).runUntilStopped\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/wsclient.go:265\ngithub.com/open-telemetry/opamp-go/client/internal.(*ClientCommon).StartConnectAndRun.func1\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/internal/clientcommon.go:197"}
{"level":"error","timestamp":"2025-01-08T05:19:57.478Z","caller":"opamp/server_client.go:216","msg":"failed to apply config","component":"opamp-server-client","error":"failed to reload config: /var/tmp/collector-config.yaml: collector failed to restart: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:\n\n* error decoding 'exporters': error reading configuration for \"clickhousetraces\": 1 error(s) decoding:\n\n* '' has invalid keys: use_new_schema","stacktrace":"github.com/SigNoz/signoz-otel-collector/opamp.(*serverClient).onRemoteConfigHandler\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/server_client.go:216\ngithub.com/SigNoz/signoz-otel-collector/opamp.(*serverClient).onMessageFuncHandler\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/opamp/server_client.go:199\ngithub.com/open-telemetry/opamp-go/client/types.CallbacksStruct.OnMessage\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/types/callbacks.go:162\ngithub.com/open-telemetry/opamp-go/client/internal.(*receivedProcessor).ProcessReceivedMessage\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/internal/receivedprocessor.go:131\ngithub.com/open-telemetry/opamp-go/client/internal.(*wsReceiver).ReceiverLoop\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/internal/wsreceiver.go:57\ngithub.com/open-telemetry/opamp-go/client.(*wsClient).runOneCycle\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/wsclient.go:243\ngithub.com/open-telemetry/opamp-go/client.(*wsClient).runUntilStopped\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/wsclient.go:265\ngithub.com/open-telemetry/opamp-go/client/internal.(*ClientCommon).StartConnectAndRun.func1\n\t/home/runner/go/pkg/mod/github.com/open-telemetry/opamp-go@v0.5.0/client/internal/clientcommon.go:197"}

when i checked with GPT got the following analysis Invalid Key in Configuration: • The configuration file (

collector-config.yaml

) contains an invalid key

use_new_schema

under the

clickhousetraces

exporter. This key is not supported by the exporter. FYI using SigNoz 0.56, deployed in Kubernetes AKS cluster with helm chart. also tried to changed the value use_new_schema to "true" but still it's not shipping the logs

clickhouselogsexporter:
        dsn: tcp://${CLICKHOUSE_USER}:${CLICKHOUSE_PASSWORD}@${CLICKHOUSE_HOST}:${CLICKHOUSE_PORT}/${CLICKHOUSE_LOG_DATABASE}
        timeout: 10s
        use_new_schema: true

Please any one help me here to troubleshoot the otelcollector, and take it back to working condition.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

Hi Team, After upgrading from 0.77.0 to 0.78.1, signoz-0 would not start up.

{"level":"fatal","timestamp":"2025-04-11T18:40:54.547Z","caller":"query-service/main.go:108","msg":"Failed to create signoz","error":"FOREIGN KEY constraint failed","stacktrace":"main.main\n\t/home/runner/work/signoz/signoz/ee/query-service/main.go:108\nruntime.main\n\t/opt/hostedtoolcache/go/1.22.12/x64/src/runtime/proc.go:271"}

We rolled back to 0.77.0, but alerting is no longer working. If I open an Alert error dialogs pop up. If I visit '/settings/channels' the text 'Something went wrong' is displayed.

"level":"ERROR","code":{"function":"<http://github.com/SigNoz/signoz/pkg/alertmanager.(*Service).SyncServers|github.com/SigNoz/signoz/pkg/alertmanager.(*Service).SyncServers>","file":"/home/runner/work/signoz/signoz/pkg/alertmanager/service.go","line":56},"msg":"failed to get alertmanager config for org","logger":"<http://github.com/SigNoz/signoz/pkg/alertmanager/signozalertmanager|github.com/SigNoz/signoz/pkg/alertmanager/signozalertmanager>","orgID":"blah-blah-blah-blah-blah","error":"sql: Scan error on column index 0, name \"id\": strconv.ParseUint: parsing \"blah-blah-blah-blah-blah\": invalid syntax"}

this is self-hosted signoz, when I try to test the email notification channel, show error

After updating some helm-values of our helm-installation of signoz, we got the the following stacktrace: