Hello, with every commit to the SigNoz Helm chart, the migration job is being deleted, which causes ...

Vinod Ahlawat

09/16/2024, 2:01 PM

Hello, with every commit to the SigNoz Helm chart, the migration job is being deleted, which causes the OTEL Collector pods to fail and enter an

Init:CrashLoopBackOff

state. Once I redeploy, it works fine. Any idea why this is happening?

Srikanth Chekuri

09/17/2024, 5:17 AM

How many shards do you have?

Vinod Ahlawat

09/17/2024, 7:14 AM

@Srikanth Chekuri I'm using default shards count which is 1. I did try earlier with 2 & 3 also, but same issue.

Srikanth Chekuri

09/17/2024, 5:29 PM

Please share more details about the migration job failure logs

Vinod Ahlawat

09/20/2024, 1:29 PM

Sure, I will share the more details

Vinod Ahlawat

09/30/2024, 5:24 PM

Below are the logs - @Srikanth Chekuri

# k get pod -n signoz | grep -i signoz-otel-collector-796f7486c6-2sb42

signoz-otel-collector-796f7486c6-2sb42           0/1     Init:CrashLoopBackOff   5 (60s ago)   13m

# k logs signoz-otel-collector-796f7486c6-2sb42 -n signoz

Defaulted container "collector" out of: collector, signoz-otel-collector-migrate-init (init)

Error from server (BadRequest): container "collector" in pod "signoz-otel-collector-796f7486c6-2sb42" is waiting to start: PodInitializing

# k logs signoz-otel-collector-796f7486c6-2sb42 -n signoz -c signoz-otel-collector-migrate-init

Error from server (NotFound): jobs.batch "signoz-schema-migrator-init" not found

# k get job -n signoz

No resources found in signoz namespace.

10 Views

Open in Slack

Previous Next

SigNoz Community

SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.