This message was deleted.

Abel - were you able to check if the PV increase was successful. Can you share what logs are you seeing now after PV increase?

Sure, I'm getting:

exporterhelper/retry_sender.go:177    Exporting failed. Will retry the request after interval.    {"kind": "exporter", "data_type": "metrics", "name": "clickhousemetricswrite", "error": "code: 243, message: Cannot reserve 1.00 MiB, not enough space", "interval": "0s"}`

In the

signoz-otel-collector

pod and also this one:

Could not write a batch of spans to model table:     {"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "error": "code: 243, m │
│ essage: Cannot reserve 2.76 MiB, not enough space"}                                                                                                                                                                                │
│ <http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).WriteBatchOfSpans|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).WriteBatchOfSpans>                                                                                                                          │
│     /home/runner/work/signoz-otel-collector/signoz-otel-collector/exporter/clickhousetracesexporter/writer.go:415                                                                                                                  │
│ <http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*storage).pushTraceData|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*storage).pushTraceData>                                                                                                                                 │
│     /home/runner/work/signoz-otel-collector/signoz-otel-collector/exporter/clickhousetracesexporter/clickhouse_exporter.go:409                                                                                                     │
│ <http://go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesRequest).Export|go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesRequest).Export>                                                                                                                                                      │
│     /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/traces.go:60                                                                                                                             │
│ <http://go.opentelemetry.io/collector/exporter/exporterhelper.(*timeoutSender).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*timeoutSender).send>                                                                                                                                                        │
│     /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/timeout_sender.go:41                                                                                                                     │
│ <http://go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send>                                                                                                                                                          │
│     /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/retry_sender.go:138                                                                                                                      │
│ <http://go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesExporterWithObservability).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*tracesExporterWithObservability).send>                                                                                                                                      │
│     /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/traces.go:177                                                                                                                            │
│ <http://go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1|go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1>                                                                                                                                                   │
│     /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/queue_sender.go:126                                                                                                                      │
│ <http://go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1|go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1>                                                                                                                                   │
│     /home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/internal/bounded_memory_queue.go:52

I confirm

pv

was successfully increased. I also see an increased memory usage in this pod

~5Gi

reduce the retention period