Hi, im just trying to upgrade to SigNoz 0.47 from 0.46.1 and migration is crashing with the following error: {"level":"error","timestamp":"2024-06-05T144740.007Z","caller":"migrationmanager/manager.go:81","msg":"Failed to run migrations for migrator","component":"migrationmanager","migrator":"logs","error":"failed to create database, err: code: 159, message: Distributed DDL task /clickhouse/task_queue/ddl/query-0000000195 is not finished on 1 of 1 hosts (0 of them are currently executing the task, 0 are inactive). They are going to execute the query in background. Was waiting for 180.882823243 seconds, which is longer than distributed_ddl_task_timeout","stacktrace":"github.com/SigNoz/signoz-otel-collector/migrationmanager.(*MigrationManager).Migrate\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/migrationmanager/manager.go81\nmain.main\n\t/home/runner/work/signoz otel collector/signoz otel collector/cmd/signozschemamigrator/migrate.go126\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.10/x64/src/runtime/proc.go:267"} {"level":"fatal","timestamp":"2024-06-05T144740.007Z","caller":"signozschemamigrator/migrate.go:128","msg":"Failed to run migrations","component":"migrate cli","error":"failed to create database, err: code: 159, message: Distributed DDL task /clickhouse/task_queue/ddl/query-0000000195 is not finished on 1 of 1 hosts (0 of them are currently executing the task, 0 are inactive). They are going to execute the query in background. Was waiting for 180.882823243 seconds, which is longer than distributed_ddl_task_timeout","stacktrace":"main.main\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/migrate.go128\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.10/x64/src/runtime/proc.go267"}

Are you on the

main

branch? I’m having similar issues currently. the syslog configuration is causing it to crash, and it just appears that the migrations are not creating any database tables in the clickhouse db. I just saw there was another merged PR to the main branch 2hrs ago, you might try. https://github.com/SigNoz/signoz/pull/5151

Yes, we are using official releases

For me, Docker standalone, AWS EC2 ubuntu Ubuntu 20.04.3 LTS. If i watch the docker logs

docker logs -f signoz-clickhouse

I see what looks like failed migrations during the setup.