https://signoz.io logo
Join Slack
Powered by
Hello team ! I have an issue with the docker insta...
# support
e
Hello team ! I have an issue with the docker installation, the clickhouse-setup_otel-collector_1 collector crash (logs in thread). Any idea how to fix it ?
Copy code
2022-12-05T09:55:38.135Z info service/telemetry.go:110 Setting up own telemetry...

2022-12-05T09:55:38.136Z info service/telemetry.go:140 Serving Prometheus metrics {"address": "0.0.0.0:8888", "level": "basic"}

2022-12-05T09:55:38.136Z info components/components.go:30 Stability level of component is undefined {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "stability": "undefined"}

2022-12-05T09:55:38.142Z info clickhousetracesexporter/clickhouse_factory.go:82 Running migrations from path:  {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "test": "/migrations"}

2022-12-05T09:55:38.163Z info clickhousetracesexporter/clickhouse_factory.go:94 Clickhouse Migrate finished {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "error": "no change"}

2022-12-05T09:55:38.195Z info components/components.go:30 Stability level of component is undefined {"kind": "processor", "name": "signozspanmetrics/prometheus", "pipeline": "traces", "stability": "undefined"}

2022-12-05T09:55:38.196Z info signozspanmetricsprocessor/processor.go:104 Building signozspanmetricsprocessor {"kind": "processor", "name": "signozspanmetrics/prometheus", "pipeline": "traces"}

2022-12-05T09:55:38.196Z info components/components.go:30 Stability level of component is undefined {"kind": "exporter", "data_type": "metrics", "name": "clickhousemetricswrite", "stability": "undefined"}

time="2022-12-05T09:55:38Z" level=info msg="Executing:\nCREATE DATABASE IF NOT EXISTS signoz_metrics\n" component=clickhouse

time="2022-12-05T09:55:38Z" level=info msg="Executing:\nCREATE TABLE IF NOT EXISTS signoz_metrics.samples_v2 (\n\t\t\tmetric_name LowCardinality(String),\n\t\t\tfingerprint UInt64 Codec(DoubleDelta, LZ4),\n\t\t\ttimestamp_ms Int64 Codec(DoubleDelta, LZ4),\n\t\t\tvalue Float64 Codec(Gorilla, LZ4)\n\t\t)\n\t\tENGINE = MergeTree\n\t\t\tPARTITION BY toDate(timestamp_ms / 1000)\n\t\t\tORDER BY (metric_name, fingerprint, timestamp_ms)\n" component=clickhouse

time="2022-12-05T09:55:38Z" level=info msg="Executing:\nSET allow_experimental_object_type = 1\n" component=clickhouse

time="2022-12-05T09:55:38Z" level=info msg="Executing:\nCREATE TABLE IF NOT EXISTS signoz_metrics.time_series_v2 (\n\t\t\tmetric_name LowCardinality(String),\n\t\t\tfingerprint UInt64 Codec(DoubleDelta, LZ4),\n\t\t\ttimestamp_ms Int64 Codec(DoubleDelta, LZ4),\n\t\t\tlabels String Codec(ZSTD(5))\n\t\t)\n\t\tENGINE = ReplacingMergeTree\n\t\t\tPARTITION BY toDate(timestamp_ms / 1000)\n\t\t\tORDER BY (metric_name, fingerprint)\n" component=clickhouse

time="2022-12-05T09:55:38Z" level=info msg="Executing:\nALTER TABLE signoz_metrics.time_series_v2 DROP COLUMN IF EXISTS labels_object\n" component=clickhouse

2022-12-05T09:55:38.220Z info clickhouselogsexporter/exporter.go:247 Running migrations from path:  {"kind": "exporter", "data_type": "logs", "name": "clickhouselogsexporter", "test": "/logsmigrations"}

Error: cannot build pipelines: failed to create "clickhouselogsexporter" exporter, in pipeline "logs": cannot configure clickhouse logs exporter: clickhouse Migrate failed to run, error: Dirty database version 2. Fix and force version.

2022/12/05 09:55:38 application run finished with error: cannot build pipelines: failed to create "clickhouselogsexporter" exporter, in pipeline "logs": cannot configure clickhouse logs exporter: clickhouse Migrate failed to run, error: Dirty database version 2. Fix and force version.
I tried to clear everything, even run 
docker prune system --all
with no luck šŸ˜ž
n
Can you give us an idea about what machine you are running it on?
e
Sure, itā€™s a 
MacBook Pro (15-inch, 2019), 2,6 GHz 6-Core Intel Core i7, 32 GB 2400 MHz DDR4
with Docker 
Docker version 20.10.13, build a224086
n
Interesting, from the docs the 
docker prune system --all
doesnā€™t delete the volumes automatically. Try this one
Copy code
docker system prune -a --volumes
Also, did you make any changes before arriving in this state or it started appearing the first time you installed it?
e
I already deleted all volumes as well, does not fix the error. It appeared the first time I installed it .
n
@Prashant Shahi any ideas on this. Also, @Emmanuel Iturbide are you using the main branch of signoz?
e
Iā€™m using the main branch, but tried as well with some tags, same error.
p
@Emmanuel Iturbide By 
main
you mean the 
develop
branch?
e
By 
main
I mean 
main
Git log :
Copy code
commit 18d80d47e5ad3f68220e84e509fa493f7ff20a26 (HEAD -> main, origin/main)
Merge: 78d23775 8e552282
Author: Ankit Nayan <ankit@signoz.io>
Date:   Tue Nov 29 17:36:31 2022 +0530

    Merge pull request #1776 from SigNoz/release/v0.11.4

    Release/v0.11.4

commit 8e5522820ca611d3a72a81b638f15f7b90abfccc (tag: v0.11.4, tag: v0.11, origin/release/v0.11.4, origin/release/v0.11)
Author: Prashant Shahi <prashant@signoz.io>
Date:   Tue Nov 29 17:12:17 2022 +0530

    chore: šŸ“Œ pin versions: SigNoz 0.11.4

    Signed-off-by: Prashant Shahi <prashant@signoz.io>
p
I asked that because dirty migration issue was present in 
develop
can you bring down the SigNoz cluster and clean up the data?
Copy code
cd signoz

make clear-standalone-data
Now, bring it back up.
e
Now it works !
p
Good to know that šŸ™‚
e
What was the issue ? Some ā€œcorrupt filesā€ locally ?
p
it could happen if you had run signoz from 
develop
before.
@Vishal Sharma might have better idea on that.
v
@Emmanuel Iturbide The error occurs when clickhouse DB migrations fail. Did you fresh install or was upgrading from older version?
199 Views
Open in Slack
PreviousNext
Powered by