I am just upgrading from 0.41.0 to 0.42.0. For one cluster it did go without any issue but now have one cluster where

monitor-signoz-otel-collector-cc57b8f4-gffkk

and

monitor-signoz-otel-collector-metrics-799fd94b69-fz9xr

get in a

Init:CrashLoopBackOff

because

Error from server (NotFound): jobs.batch "monitor-signoz-schema-migrator-upgrade" not found

; any guidenance how to fix this?

@Dave Smits please i dont get what you did here, i am using terraform helm deployment

resource "helm_release" "signoz" {
  name         = "signoz"
  chart        = "signoz"
  repository   = "<https://charts.signoz.io>"
  version      = "0.39.1"
  namespace    = kubernetes_namespace.platform.id
  timeout      = "800"
  values       = [data.template_file.signoz-values.rendered]
  force_update = true
}

I manually scaled down the monitor-signoz-otel-collector and monitor-signoz-otel-collector-metrics deployment to get the schema migrator getting started. Helm was waiting for them those pods i scaled down to become ready, which they never going to get because they crashed as the schema migrator was missing

i just did the same now, the schema migrator didnt start

kubectl scale deploy signoz-otel-collector-metrics -n platform --replicas=1

I scaled them to 0

okay

hi i have tried scaling down to 0 and back up but it did not trigger the signoz-schema-migrator-upgrade, how can i trigger that manually?

What I think I did was editing the configuration to 0 instead of scaling it down

im looking at the job/signoz-schema-migrator-upgrade that is what creates the migrator but it has exceeded the retries

The whole point of scaling down is so you have 0 pods that are in not ready state. So maybe you have other pods as well that are not yet ready where helm is waiting on before creating the job.

scaling both collector/collector-metric only make it wait for "schema-migrator-upgrade"

// want to update signoz, add wait=false in your terraform helm code and also scale down the 0 the resources to be updated