This message was deleted.

Hi Team, I've done a fresh setup of SigNoz with clickhouse on EKS with 2 shards and 2 replicas (https://signoz.io/docs/operate/clickhouse/distributed-clickhouse/#kubernetes-installation), Now, whenever I do a helm upgrade, the signoz-schema-migrator-sync job runs and fails few times due to table not found errors then automatically succeeds.

Error: code: 60, message: There was an error on [chi-signoz-clickhouse-cluster-1-1:9000]: Code: 60. DB::Exception: Could not find table: time_series_v4. (UNKNOWN_TABLE) (version 24.1.2.5 (official build))

In clickhouse logs as well I see missing table errors. error logs of chi-signoz-clickhouse-cluster-0-0-0:

"message":"Code: 60. DB::Exception: Received from chi-signoz-clickhouse-cluster-1-1:9000. DB::Exception: Table signoz_metrics.samples_v4 does not exist.

"message":"Code: 60. DB::Exception: Received from chi-signoz-clickhouse-cluster-1-1:9000. DB::Exception: Table signoz_metrics.samples_v2 does not exist.

Then while setting the retention on UI, it just gets stuck and in the logs of the query service I see it is not able to do GetTTL from clickhouse:

"msg":"http: panic serving 10.10.249.136:55804: runtime error: invalid memory address or nil pointer dereference\ngoroutine 684 [running]:\nnet/http.(*conn).serve.func1()\n\t/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.7.linux-amd64/src/net/http/server.go:1903 +0xbe\npanic({0x22593a0?, 0x4155d20?})\n\t/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.22.7.linux-amd64/src/runtime/panic.go:770 +0x132\<http://ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).GetTTL(0xc00011f688|ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).GetTTL(0xc00011f688>, {0x2f132b8, 0xc000d074a0}

I cleared the SQLite db table as well, but it is still stuck. (https://signoz.io/docs/faqs/troubleshooting/#i-am-trying-to-change-the-retention-period-of-traces-but-the-process-gets-stuck-everytime) Am I missing something wrt to the db schemas? Is anyone able to make it work with the latest helm chart appVersion=0.73.0?

This message was deleted.

Hey folks - Docker standalone is failing due to an issue encountered with the ClickHouse Docker container. When attempting to run the ClickHouse Docker container on Oracle Linux 8 with Docker version 24.0.7 and ClickHouse server version 24.1.2-alpine, the following error is encountered:

Processing configuration file '/etc/clickhouse-server/config.xml'.

Poco::Exception. Code: 1000, e.code() = 13, Access to file denied: /etc/clickhouse-server/config.xml

Full log dump: https://gitlab.com/-/snippets/3690972 I attempted to resolve the issue by creating the path /etc/clickhouse-server/ with full permissions . Despite trying to bring up the Docker Compose setup using the root user, the ClickHouse Docker container running on Oracle Linux 8 with Docker version 24.0.7 and ClickHouse server version 24.1.2-alpine still encounters the following error: No luck This issue seems to be related to file permission or access control within the Docker container environment. Any insights or suggestions on how to resolve this issue would be greatly appreciated. Please let me know if further information is required.

This message was deleted.

Hi folks, Trying to connect self hosted signoz from spring boot application using below command java -javaagent:./agent/opentelemetry-javaagent.jar \ -Dotel.exporter.otlp.endpoint=http://localhost:4317 \ -Dotel.resource.attributes=service.name=app1 \ -jar ./target/app1-0.0.1-SNAPSHOT.jar and getting below error C[otel.javaagent 2024-05-06 210336:156 +0530] [OkHttp http://localhost:4317/...] ERROR io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export metrics. The request could not be executed. Full error message: unexpected end of stream on http://localhost:4317/... java.io.IOException: unexpected end of stream on http://localhost:4317/... at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:210) at okhttp3.internal.connection.Exchange.readResponseHeaders(Exchange.kt:110) at okhttp3.internal.http.CallServerInterceptor.intercept(CallServerInterceptor.kt:93) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.kt:34) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.cache.CacheInterceptor.intercept(CacheInterceptor.kt:95) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.kt:83) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.kt:76) at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.kt:109) at okhttp3.internal.connection.RealCall.getResponseWithInterceptorChain$okhttp(RealCall.kt:201) at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:517) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:858) Caused by: java.io.EOFException: \n not found: limit=15 content=000006040000000000000500004000… at okio.RealBufferedSource.readUtf8LineStrict(RealBufferedSource.kt:339) at okhttp3.internal.http1.HeadersReader.readLine(HeadersReader.kt:29) at okhttp3.internal.http1.Http1ExchangeCodec.readResponseHeaders(Http1ExchangeCodec.kt:180) ... 16 more can someone please advise

Currently I'm trying to set mTLS between two OTEL collectors one is running on docker and another one is running on Kubernetes. While setting up K8s I'm getting this following error @Prashant Shahi

2024-06-18T06:49:16.787Z	info	exporterhelper/retry_sender.go:177	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"", "interval": "13.402682275s"}
2024-06-18T06:49:30.191Z	info	exporterhelper/retry_sender.go:177	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"", "interval": "20.649604662s"}

But it works well with the other Otel collector. Here is the command which i'm using to setup K8s-Infra. Also I'm using Secrets for TLS certs in k8s.

helm --namespace k8s-infra install my-release signoz/k8s-infra \
  --set otelCollectorEndpoint=<https://example.dev:4317> \
  --set otelInsecure=false \
  --set global.clusterName=k04-admin@kubernetes \
  --set tls.enabled=true \
  --set tls.secretName=ss-dev-new \
  --set tls.caCert=rootCA.crt \
  --set tls.tlsCert=monitor.crt \
  --set tls.tlsKey=monitor.key

Hello Everyone, When i invite member to signoz, then why mail has not been sent to them?

This message was deleted.

This message was deleted.