https://signoz.io logo
Join Slack
Powered by
Trying to migrate from `v0.87.0` to `v0.88.0` but ...
# support
c
Trying to migrate from 
v0.87.0
to 
v0.88.0
but both migration scripts are failling:
Untitled.json
Untitled.json
And yes, I have the correct exporters enabled:
Copy code
pipelines:
        logs:
          exporters:
          - clickhouselogsexporter
          - metadataexporter
          processors:
          - batch
          receivers:
          - otlp
          - httplogreceiver/heroku
          - httplogreceiver/json
        metrics:
          exporters:
          - clickhousemetricswrite
          - metadataexporter
          - signozclickhousemetrics
          processors:
          - batch
          receivers:
          - otlp
        traces:
          exporters:
          - clickhousetraces
          - metadataexporter
          processors:
          - signozspanmetrics/delta
          - batch
          receivers:
          - otlp
          - jaeger
As well as processors:
Copy code
processors:
      batch:
        send_batch_size: 50000
        timeout: 1s
      signozspanmetrics/delta:
        aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA
        dimensions:
        - default: default
          name: service.namespace
        - default: default
          name: deployment.environment
        - name: signoz.collector.id
        dimensions_cache_size: 100000
        latency_histogram_buckets:
        - 100us
        - 1ms
        - 2ms
        - 6ms
        - 10ms
        - 50ms
        - 100ms
        - 250ms
        - 500ms
        - 1000ms
        - 1400ms
        - 2000ms
        - 5s
        - 10s
        - 20s
        - 40s
        - 60s
        metrics_exporter: clickhousemetricswrite, signozclickhousemetrics
The first migration job
Untitled.json
s
c
hmmm it's a very long list 😅 do I include the entire thing?
I'm also trying to run all these migration scripts before I upgrade to 0.88. That's how it should be, correct? I shouldn't be trying to run 0.88 first?
s
That's how it should be, correct?
Yes, so your existing dashboards and alerts are migrated with sqlite migration
do I include the entire thing?
Yes, please write a simple script to prepare a map
👍 1
c
So I tried putting all of the metrics into 
NOT_FOUND_METRICS_MAP
e.g. 
ClickHouseErrorMetric_CANNOT_READ_FROM_FILE_DESCRIPTOR=ClickHouseErrorMetric_CANNOT_READ_FROM_FILE_DESCRIPTOR,ClickHouseErrorMetric_QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW=ClickHouseErrorMetric_QUERY_IS_NOT_SUPPORTED_IN_MATERIALIZED_VIEW,ClickHouseErrorMetric_RECEIVED_ERROR_FROM_REMOTE_IO_SERVER=ClickHouseErrorMetric_RECEIVED_ERROR_FROM_REMOTE_IO_SERVER,chi_clickhouse_event_ThreadPoolReaderPageCacheMissElapsedMicroseconds=chi_clickhouse_event_ThreadPoolReaderPageCacheMissElapsedMicroseconds...
But it actually failed with this error now:
Copy code
exec /usr/local/bin/migrate: argument list too long
stream closed EOF for cluster-addon-signoz/signoz-data-migration-job-c4bnn (migration)
Ok, so ended up wiping all my metrics and trying everything from scratch. What ended up happening is that somewhere between helm chart upgrades, there used to be a 
signoz-otel-collector-metrics
that somehow ended up orphan and not tracked by ArgoCD, which was pushing all those metrics with the underscore
Deleted that deployment, wiped clickhouse again, re-did the migrations, and so far so good
3 Views
Open in Slack
PreviousNext
Powered by