Hi Team, After upgrading from 0.77.0 to 0.78.1, signoz-0 would not start up.

{
  "level": "fatal",
  "timestamp": "2025-04-11T18:40:54.547Z",
  "caller": "query-service/main.go:108",
  "msg": "Failed to create signoz",
  "error": "FOREIGN KEY constraint failed",
  "stacktrace": "main.main\n\t/home/runner/work/signoz/signoz/ee/query-service/main.go:108\nruntime.main\n\t/opt/hostedtoolcache/go/1.22.12/x64/src/runtime/proc.go:271"
}

We rolled back to 0.77.0, but alerting is no longer working. If I open an Alert error dialogs pop up. If I visit '/settings/channels' the text 'Something went wrong' is displayed.

"level":"ERROR","code":{"function":"<http://github.com/SigNoz/signoz/pkg/alertmanager.(*Service).SyncServers|github.com/SigNoz/signoz/pkg/alertmanager.(*Service).SyncServers>","file":"/home/runner/work/signoz/signoz/pkg/alertmanager/service.go","line":56},"msg":"failed to get alertmanager config for org","logger":"<http://github.com/SigNoz/signoz/pkg/alertmanager/signozalertmanager|github.com/SigNoz/signoz/pkg/alertmanager/signozalertmanager>","orgID":"blah-blah-blah-blah-blah","error":"sql: Scan error on column index 0, name \"id\": strconv.ParseUint: parsing \"blah-blah-blah-blah-blah\": invalid syntax"}

@Vibhu Pandey ^

Suspect that the path forward is upgrading to 0.78.1. The first attempt failed with:

{
  "level": "fatal",
  "timestamp": "2025-04-11T18:40:54.547Z",
  "caller": "query-service/main.go:108",
  "msg": "Failed to create signoz",
  "error": "FOREIGN KEY constraint failed",
  "stacktrace": "main.main\n\t/home/runner/work/signoz/signoz/ee/query-service/main.go:108\nruntime.main\n\t/opt/hostedtoolcache/go/1.22.12/x64/src/runtime/proc.go:271"
}

So we rolled back to 0.77.0, but the alert tables schema likely remains v0.78.x and therefore incompatible with v0.77.0. I think we can ignore the alerting issues and focus on how to upgrade to 0.78.1 and resolve :

FOREIGN KEY constraint failed stacktrace

Upgrading to 0.78.0 also gives the

FOREIGN KEY constraint failed

error

Hi Team, we are requesting support to help troubleshoot the below fatal-error that stops signoz:v0.78.1 starting up. Are there any troubleshooting steps that we can perform to help diagnose and resolve this problem?

35:10.0,"Container image "<http://docker.io/signoz/signoz:v0.78.1|docker.io/signoz/signoz:v0.78.1>" already present on machine",,,otel-deployment
35:10.3,[Deprecated] env STORAGE is deprecated and scheduled for removal. Please use SIGNOZ_TELEMETRYSTORE_PROVIDER instead.,signoz,signoz-0,otel-agent
35:10.3,[Deprecated] env ClickHouseUrl is deprecated and scheduled for removal. Please use SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN instead.,signoz,signoz-0,otel-agent
35:10.3,[Deprecated] flag --config is deprecated for passing prometheus config. The flag will be used for passing the entire SigNoz config. More details can be found at <https://github.com/SigNoz/signoz/issues/6805.,signoz,signoz-0,otel-agent>
35:10.3,"{"timestamp":"2025-04-11T18:35:10.316158489Z","level":"INFO","code":{"function":"<http://github.com/SigNoz/signoz/pkg/signoz.New|github.com/SigNoz/signoz/pkg/signoz.New>","file":"/home/runner/work/signoz/signoz/pkg/signoz/signoz.go","line":44},"msg":"starting signoz","version":"v0.78.1","variant":"enterprise","commit":"8d63070","branch":"v0.78.1","go":"go1.22.12","timestamp":"2025-04-09T09:58:33Z"}",signoz,signoz-0,otel-agent
35:10.3,"{"timestamp":"2025-04-11T18:35:10.316395183Z","level":"INFO","code":{"function":"<http://github.com/SigNoz/signoz/pkg/sqlstore/sqlitesqlstore.New|github.com/SigNoz/signoz/pkg/sqlstore/sqlitesqlstore.New>","file":"/home/runner/work/signoz/signoz/pkg/sqlstore/sqlitesqlstore/provider.go","line":45},"msg":"connected to sqlite","logger":"<http://github.com/SigNoz/signoz/pkg/sqlitesqlstore|github.com/SigNoz/signoz/pkg/sqlitesqlstore>","path":"/var/lib/signoz/signoz.db"}",signoz,signoz-0,otel-agent
35:10.3,"{"timestamp":"2025-04-11T18:35:10.316509227Z","level":"ERROR","code":{"function":"<http://github.com/prometheus/prometheus/promql.NewActiveQueryTracker|github.com/prometheus/prometheus/promql.NewActiveQueryTracker>","file":"/home/runner/go/pkg/mod/github.com/prometheus/prometheus@v0.300.1/promql/query_logger.go","line":137},"msg":"Failed to create directory for logging active queries","logger":"<http://github.com/SigNoz/signoz/pkg/prometheus/clickhouseprometheus|github.com/SigNoz/signoz/pkg/prometheus/clickhouseprometheus>"}",signoz,signoz-0,otel-agent
35:10.3,"{"timestamp":"2025-04-11T18:35:10.317134069Z","level":"INFO","code":{"function":"<http://github.com/SigNoz/signoz/pkg/sqlmigrator.(*migrator).Migrate|github.com/SigNoz/signoz/pkg/sqlmigrator.(*migrator).Migrate>","file":"/home/runner/work/signoz/signoz/pkg/sqlmigrator/migrator.go","line":43},"msg":"starting sqlstore migrations","logger":"<http://github.com/SigNoz/signoz/pkg/sqlmigrator|github.com/SigNoz/signoz/pkg/sqlmigrator>","dialect":"sqlite"}",signoz,signoz-0,otel-agent
35:10.4,"{"timestamp":"2025-04-11T18:35:10.354300188Z","level":"INFO","code":{"function":"<http://github.com/SigNoz/signoz/pkg/sqlmigrator.(*migrator).Lock|github.com/SigNoz/signoz/pkg/sqlmigrator.(*migrator).Lock>","file":"/home/runner/work/signoz/signoz/pkg/sqlmigrator/migrator.go","line":90},"msg":"acquired migration lock","logger":"<http://github.com/SigNoz/signoz/pkg/sqlmigrator|github.com/SigNoz/signoz/pkg/sqlmigrator>","dialect":"sqlite"}",signoz,signoz-0,otel-agent
35:10.5,"2025/04/11 18:35:10 WARN: bun: Organization.IsAnonymous has unknown tag option: "CHECK(is_anonymous IN (0"",signoz,signoz-0,otel-agent
35:10.5,"2025/04/11 18:35:10 WARN: bun: Organization.IsAnonymous has unknown tag option: "1))"",signoz,signoz-0,otel-agent
35:10.5,"2025/04/11 18:35:10 WARN: bun: Organization.HasOptedUpdates has unknown tag option: "CHECK(has_opted_updates IN (0"",signoz,signoz-0,otel-agent
35:10.5,"2025/04/11 18:35:10 WARN: bun: Organization.HasOptedUpdates has unknown tag option: "1))"",signoz,signoz-0,otel-agent
35:10.5,"{"level":"fatal","timestamp":"2025-04-11T18:35:10.529Z","caller":"query-service/main.go:108","msg":"Failed to create signoz","error":"FOREIGN KEY constraint failed","stacktrace":"main.main\n\t/home/runner/work/signoz/signoz/ee/query-service/main.go:108\nruntime.main\n\t/opt/hostedtoolcache/go/1.22.12/x64/src/runtime/proc.go:271"}",signoz,signoz-0,otel-agent

Hey @Al, Can you exec into your sqlite db and remove any hanging entries in reset_password_request table? You can truncate the table.

DELETE FROM reset_password_request

@Vibhu Pandey Success! Thank you for your support. We are back up and running. 🙌

The same fix for you @Eric Therrien should work out