https://signoz.io logo
Join SlackCommunities
Powered by
Hi, I have an issue after upgrade from 0.42.0 to 0...
# support
e
Hi, I have an issue after upgrade from 0.42.0 to 0.80.0 ( Docker Standalone ). This is after a clean reinstall
Copy code
docker logs --timestamps signoz-clickhouse --tail 50

2025-04-23T08:02:52.224813178Z {"date_time":"1745395372.223565","thread_name":"TCPServerConnection ([#9])","thread_id":"735","level":"Error","query_id":"69062364-c615-4d40-af18-f54763c79937","logger_name":"TCPHandler","message":"Code: 60. DB::Exception: Table signoz_traces.distributed_signoz_error_index_v2 does not exist. Maybe you meant signoz_traces.distributed_signoz_index_v3?. (UNKNOWN_TABLE), Stack trace (when copying this message, always include the lines below):\n\n0. DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000c800f1b in \/usr\/bin\/clickhouse\n1. DB::Exception::Exception<String, String, String, String>(int, FormatStringHelperImpl<std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type, std::type_identity<String>::type>, String&&, String&&, String&&, String&&) @ 0x00000000072ccc41 in \/usr\/bin\/clickhouse\n2. DB::IDatabase::getTable(String const&, std::shared_ptr<DB::Context const>) const @ 0x00000000109af4a1 in \/usr\/bin\/clickhouse\n3. DB::DatabaseCatalog::getTableImpl(DB::StorageID const&, std::shared_ptr<DB::Context const>, std::optional<DB::Exception>*) const @ 0x0000000010d16f36 in \/usr\/bin\/clickhouse\n4. DB::DatabaseCatalog::getTable(DB::StorageID const&, std::shared_ptr<DB::Context const>) const @ 0x0000000010d20a89 in \/usr\/bin\/clickhouse\n5. DB::InterpreterInsertQuery::getTable(DB::ASTInsertQuery&) @ 0x00000000114bb434 in \/usr\/bin\/clickhouse\n6. DB::InterpreterInsertQuery::execute() @ 0x00000000114bf337 in \/usr\/bin\/clickhouse\n7. DB::executeQueryImpl(char const*, char const*, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum, DB::ReadBuffer*) @ 0x0000000011904974 in \/usr\/bin\/clickhouse\n8. DB::executeQuery(String const&, std::shared_ptr<DB::Context>, DB::QueryFlags, DB::QueryProcessingStage::Enum) @ 0x00000000118ff77a in \/usr\/bin\/clickhouse\n9. DB::TCPHandler::runImpl() @ 0x000000001291be29 in \/usr\/bin\/clickhouse\n10. DB::TCPHandler::run() @ 0x0000000012933eb9 in \/usr\/bin\/clickhouse\n11. Poco::Net::TCPServerConnection::start() @ 0x00000000153a5a72 in \/usr\/bin\/clickhouse\n12. Poco::Net::TCPServerDispatcher::run() @ 0x00000000153a6871 in \/usr\/bin\/clickhouse\n13. Poco::PooledThread::run() @ 0x000000001549f047 in \/usr\/bin\/clickhouse\n14. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001549d67d in \/usr\/bin\/clickhouse\n15. ? @ 0x00007f728b336609\n16. ? @ 0x00007f728b25b353\n","source_file":"src\/Server\/TCPHandler.cpp; void DB::TCPHandler::runImpl()","source_line":"686"}
Copy code
docker logs --timestamps schema-migrator-sync --tail 50

2025-04-23T08:03:48.825392946Z {"L":"info","timestamp":"2025-04-23T08:03:48.825Z","C":"schema_migrator/manager.go:450","M":"Fetching non-finished DDLs from distributed DDL queue"}
2025-04-23T08:03:49.123973900Z {"L":"info","timestamp":"2025-04-23T08:03:49.123Z","C":"schema_migrator/manager.go:473","M":"No pending DDLs found in distributed DDL queue"}
2025-04-23T08:03:49.124003050Z {"L":"info","timestamp":"2025-04-23T08:03:49.123Z","C":"schema_migrator/manager.go:862","M":"Inserting migration entry","query":"INSERT INTO signoz_traces.distributed_schema_migrations_v2 (migration_id, status, created_at) VALUES (1000, 'in-progress', '2025-04-23 08:03:49')"}
2025-04-23T08:03:49.126573985Z {"L":"info","timestamp":"2025-04-23T08:03:49.126Z","C":"schema_migrator/manager.go:919","M":"Running operation","sql":"ALTER TABLE signoz_traces.root_operations ON CLUSTER cluster MODIFY QUERY SELECT DISTINCT\n\t\t\t\t\t\t\tname,\n\t\t\t\t\t\t\tresource_string_service$$name as serviceName\n\t\t\t\t\t\tFROM signoz_traces.signoz_index_v3\n\t\t\t\t\t\tWHERE parent_span_id = ''"}
2025-04-23T08:03:49.183303749Z {"L":"info","timestamp":"2025-04-23T08:03:49.183Z","C":"schema_migrator/manager.go:868","M":"Updating migration entry","query":"ALTER TABLE signoz_traces.schema_migrations_v2 ON CLUSTER cluster UPDATE status = $1, error = $2, updated_at = $3 WHERE migration_id = $4","status":"failed","error":"code: 60, message: There was an error on [clickhouse:9000]: Code: 60. DB::Exception: Could not find table: root_operations. (UNKNOWN_TABLE) (version 24.1.2.5 (official build))","migration_id":1000}
2025-04-23T08:03:49.241970782Z Error: code: 60, message: There was an error on [clickhouse:9000]: Code: 60. DB::Exception: Could not find table: root_operations. (UNKNOWN_TABLE) (version 24.1.2.5 (official build))
2025-04-23T08:03:49.243499928Z Usage:
2025-04-23T08:03:49.243513119Z   signoz-schema-migrator sync [flags]
2025-04-23T08:03:49.243516438Z
...
n
By clean reinstall did you also delete the clickhouse volume mount ?
e
yes, I deleted all docker volumes / images
Services are partially working after a lot of manual databases updates / inserts like these below - but it still gives errors. But it feels like there shouldn't be any errors after a clean reinstall.
Copy code
ALTER TABLE signoz_traces.distributed_tag_attributes_v2
    ADD COLUMN `tag_key` String,
    ADD COLUMN `tag_type` String,
    ADD COLUMN `tag_data_type` String,
    ADD COLUMN `string_value` String,
    ADD COLUMN `number_value` Float64,
    ADD COLUMN `unix_milli` UInt64
Copy code
CREATE TABLE signoz_traces.distributed_dependency_graph_minutes_v2
(
    `trace_id` String,
    `parent_id` String,
    `service_name` String,
    `timestamp` DateTime,
    `duration_ms` UInt64,
    `status_code` Int32,
    `graph` String,
    `metadata` String
)
ENGINE = MergeTree
PARTITION BY toYYYYMM(timestamp)
ORDER BY (timestamp, trace_id)
I think it's actually solved now. I noticed signoz-net was running on 172.18.0.0/16 - which was an overlap with network which is used by host hosting SigNoz. signoz-init-clickhouse also refused to start properly. After adjustment to docker-compose.yaml - everything seems to work properly.
Copy code
services:
  init-clickhouse:
    !!merge <<: *common
    image: clickhouse/clickhouse-server:24.1.2-alpine
    container_name: signoz-init-clickhouse
    command:
      - bash
      - -c
      - |
        echo "Using local histogram-binary from /opt/signoz/deploy"
        cd /tmp
        cp /opt/signoz/deploy/histogram-quantile_linux_amd64.tar.gz histogram-quantile.tar.gz
        tar -xvzf histogram-quantile.tar.gz
        mv histogram-quantile /var/lib/clickhouse/user_scripts/histogramQuantile
    restart: on-failure
    volumes:
      - ../common/clickhouse/user_scripts:/var/lib/clickhouse/user_scripts/
      - /opt/signoz/deploy:/opt/signoz/deploy:ro
8 Views
Open in Slack
PreviousNext
Powered by