Hi. We tried to upgrade signoz on k8s from chart 0.43.0 to 0.45.2 using helm and the schema-migrator-upgrade pod failed to execute migrations on the database (the clickhouse database pod was recreated while the migrations were in progress). The helm upgrade:

Error: UPGRADE FAILED: post-upgrade hooks failed: 1 error occurred:
        * timed out waiting for the condition

While the schema-migrator-upgrade pod log:

2024-07-09T07:53:36.846408300Z {"level":"error","timestamp":"2024-07-09T07:53:36.846Z","caller":"migrationmanager/manager.go:81","msg":"Failed to run migrations for migrator","component":"migrationmanager","migrator":"logs","error":"clickhouse migrate failed to run, error: Dirty database version 12. Fix and force version.","stacktrace":"<http://github.com/SigNoz/signoz-otel-collector/migrationmanager.(*MigrationManager).Migrate|github.com/SigNoz/signoz-otel-collector/migrationmanager.(*MigrationManager).Migrate>\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/migrationmanager/manager.go:81\nmain.main\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/migrate.go:126\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.11/x64/src/runtime/proc.go:267"}
2024-07-09T07:53:36.846517200Z {"level":"fatal","timestamp":"2024-07-09T07:53:36.846Z","caller":"signozschemamigrator/migrate.go:128","msg":"Failed to run migrations","component":"migrate cli","error":"clickhouse migrate failed to run, error: Dirty database version 12. Fix and force version.","stacktrace":"main.main\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/migrate.go:128\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.11/x64/src/runtime/proc.go:267"}

Trying to delete the signoz_traces.schema_migrations/signoz_metrics.schema_migrations/signoz_logs.schema_migrations (as suggested in https://community-chat.signoz.io/t/16422187/i-m-trying-to-upgrade-a-cluster-i-installed-yesterday-from-v ) and upgrading again didn't help and we are stuck that both new versions of signoz otel-collector and otel-collector-metrics pods didn't start (v102.2; we still have the previous version 0.88.26 running). The query:

chi-signoz13-clickhouse-cluster-0-0-0.chi-signoz13-clickhouse-cluster-0-0.platform13.svc.cluster.local :) select * from schema_migrations

SELECT *
FROM schema_migrations

Query id: 71233133-bedd-4e08-916e-c640d432c325

┌─version─┬─dirty─┬────────────sequence─┐
│      12 │     1 │ 1720511516687366500 │
└─────────┴───────┴─────────────────────┘
┌─version─┬─dirty─┬────────────sequence─┐
│       1 │     1 │ 1720511509536275800 │
│       1 │     0 │ 1720511509925968100 │
│       2 │     1 │ 1720511509927867300 │
│       2 │     0 │ 1720511509984443400 │
│       3 │     1 │ 1720511509986195000 │
│       3 │     0 │ 1720511510157772100 │
│       4 │     1 │ 1720511510159524100 │
│       4 │     0 │ 1720511510216625700 │
│       5 │     1 │ 1720511510218281300 │
│       5 │     0 │ 1720511510769604900 │
│       6 │     1 │ 1720511510771235700 │
│       6 │     0 │ 1720511510885832700 │
│       7 │     1 │ 1720511510887398400 │
│       7 │     0 │ 1720511511056791700 │
│       8 │     1 │ 1720511511058826200 │
│       8 │     0 │ 1720511511334694000 │
│       9 │     1 │ 1720511511336217600 │
│       9 │     0 │ 1720511511447635300 │
│      10 │     1 │ 1720511511449157700 │
│      10 │     0 │ 1720511516075765100 │
│      11 │     1 │ 1720511516077423500 │
└─────────┴───────┴─────────────────────┘
┌─version─┬─dirty─┬────────────sequence─┐
│      11 │     0 │ 1720511516685619200 │
└─────────┴───────┴─────────────────────┘

23 rows in set. Elapsed: 0.002 sec. 

chi-signoz13-clickhouse-cluster-0-0-0.chi-signoz13-clickhouse-cluster-0-0.platform13.svc.cluster.local :

Please help.

Hello ! I'm having issues when sending application file logs to self-hosted signoz instance , http2:frame too large error is occuring

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

please how do i fix this errors in otel-collector?

{"level":"info","ts":1749784833.8750758,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"33.43241017s"}
{"level":"info","ts":1749784834.854519,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"10.298972847s"}
{"level":"info","ts":1749784836.09816,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"4.058984498s"}
{"level":"info","ts":1749784838.6349702,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"2.577215766s"}
{"level":"info","ts":1749784850.1593175,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"7.84091135s"}
{"level":"info","ts":1749784851.2132058,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"10.360665119s"}
{"level":"info","ts":1749784852.4822192,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"26.716444271s"}
{"level":"info","ts":1749784852.7835574,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"metrics","name":"clickhousemetricswrite","error":"context deadline exceeded","interval":"31.866769043s"}
{"level":"info","ts":1749784855.1549704,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"11.269614762s"}
{"level":"info","ts":1749784857.9503,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"37.636314979s"}
{"level":"info","ts":1749784863.0528462,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"27.286178514s"}
{"level":"info","ts":1749784865.6311586,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"38.264130446s"}
{"level":"info","ts":1749784868.0015748,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"11.122094322s"}
{"level":"info","ts":1749784871.5744464,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"10.521283723s"}
{"level":"info","ts":1749784871.6017382,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"43.461381439s"}
{"level":"info","ts":1749784876.4260118,"caller":"internal/retry_sender.go:118","msg":"Exporting failed. Will retry the request after interval.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"StatementSend:context deadline exceeded","interval":"21.042483971s"}{"level":"error","ts":1749785466.8774161,"caller":"internal/base_exporter.go:153","msg":"Exporting failed. Rejecting data.","kind":"exporter","data_type":"logs","name":"clickhouselogsexporter","error":"sending queue is full","rejected_items":36,"stacktrace":"<http://go.opentelemetry.io/collector/exporter/exporterhelper/internal|go.opentelemetry.io/collector/exporter/exporterhelper/internal>.

On the pricing page, under "Multi-tenancy", there is an "x" for the Teams plan. Can you describe exactly what that entails?

I have followed this document to reset the admin user credentials once. after that we are not able to create the new users. https://signoz.io/docs/operate/sqlite/reset-admin-password/