Currently I'm trying to set mTLS between two OTEL collectors one is running on docker and another on...
a

Anurag Vishwakarma

about 1 year ago
Currently I'm trying to set mTLS between two OTEL collectors one is running on docker and another one is running on Kubernetes. While setting up K8s I'm getting this following error @Prashant Shahi
2024-06-18T06:49:16.787Z	info	exporterhelper/retry_sender.go:177	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"", "interval": "13.402682275s"}
2024-06-18T06:49:30.191Z	info	exporterhelper/retry_sender.go:177	Exporting failed. Will retry the request after interval.	{"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: tls: failed to verify certificate: x509: certificate signed by unknown authority\"", "interval": "20.649604662s"}
But it works well with the other Otel collector. Here is the command which i'm using to setup K8s-Infra. Also I'm using Secrets for TLS certs in k8s.
helm --namespace k8s-infra install my-release signoz/k8s-infra \
  --set otelCollectorEndpoint=<https://example.dev:4317> \
  --set otelInsecure=false \
  --set global.clusterName=k04-admin@kubernetes \
  --set tls.enabled=true \
  --set tls.secretName=ss-dev-new \
  --set tls.caCert=rootCA.crt \
  --set tls.tlsCert=monitor.crt \
  --set tls.tlsKey=monitor.key
I'm trying to evaluate Signoz locally, but I can't get it to start. I'm using Docker Desktop on Wind...
b

Ben Page

about 1 year ago
I'm trying to evaluate Signoz locally, but I can't get it to start. I'm using Docker Desktop on Windows 11.
docker compose -f docker/clickhouse-setup/docker-compose.yaml up -d
[+] Running 11/11
 ✔ Network clickhouse-setup_default  Created                                                                       0.1s
 ✔ Container load-hotrod             Started                                                                       3.2s
 ✔ Container signoz-zookeeper-1      Started                                                                       3.6s
 ✔ Container hotrod                  Started                                                                       3.2s
 ✔ Container signoz-clickhouse       Healthy                                                                      34.5s
 ✘ Container otel-migrator           service "otel-collector-migrator" didn't complet...                          34.3s
 ✔ Container signoz-query-service    Created                                                                       0.2s
 ✔ Container signoz-alertmanager     Created                                                                       0.2s
 ✔ Container signoz-otel-collector   Created                                                                       0.2s
 ✔ Container signoz-logspout         Created                                                                       0.2s
 ✔ Container signoz-frontend         Created                                                                       0.2s
2024-06-07 18:12:40 {"level":"info","timestamp":"2024-06-07T23:12:40.900Z","caller":"signozschemamigrator/migrate.go:89","msg":"Setting env var SIGNOZ_CLUSTER","component":"migrate cli","cluster-name":"cluster"}
2024-06-07 18:12:40 {"level":"info","timestamp":"2024-06-07T23:12:40.900Z","caller":"signozschemamigrator/migrate.go:106","msg":"Successfully set env var SIGNOZ_CLUSTER ","component":"migrate cli","cluster-name":"cluster"}
2024-06-07 18:12:40 {"level":"info","timestamp":"2024-06-07T23:12:40.900Z","caller":"signozschemamigrator/migrate.go:111","msg":"Setting env var SIGNOZ_REPLICATED","component":"migrate cli","replication":false}
2024-06-07 18:12:40 {"level":"info","timestamp":"2024-06-07T23:12:40.906Z","caller":"migrationmanager/manager.go:76","msg":"Running migrations for all migrators","component":"migrationmanager"}
2024-06-07 18:12:40 {"level":"info","timestamp":"2024-06-07T23:12:40.906Z","caller":"migrationmanager/manager.go:78","msg":"Running migrations for logs","component":"migrationmanager","migrator":"logs"}
2024-06-07 18:12:41 {"level":"error","timestamp":"2024-06-07T23:12:41.058Z","caller":"migrationmanager/manager.go:81","msg":"Failed to run migrations for migrator","component":"migrationmanager","migrator":"logs","error":"clickhouse migrate failed to run, error: code: 1001, message: std::__1::__fs::filesystem::filesystem_error: filesystem error: in rename: Permission denied [\"/var/lib/clickhouse/store/b1a/b1a5f878-85cf-44da-bb9f-5e75e6fe6278/tmp_insert_all_1_1_0/\"] [\"/var/lib/clickhouse/store/b1a/b1a5f878-85cf-44da-bb9f-5e75e6fe6278/all_1_1_0/\"] in line 0: INSERT INTO schema_migrations (version, dirty, sequence) VALUES (1, 1, 1717801960995297780)","stacktrace":"github.com/SigNoz/signoz-otel-collector/migrationmanager.(*MigrationManager).Migrate\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/migrationmanager/manager.go:81\nmain.main\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/migrate.go:126\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.10/x64/src/runtime/proc.go:267"}
2024-06-07 18:12:41 {"level":"fatal","timestamp":"2024-06-07T23:12:41.058Z","caller":"signozschemamigrator/migrate.go:128","msg":"Failed to run migrations","component":"migrate cli","error":"clickhouse migrate failed to run, error: code: 1001, message: std::__1::__fs::filesystem::filesystem_error: filesystem error: in rename: Permission denied [\"/var/lib/clickhouse/store/b1a/b1a5f878-85cf-44da-bb9f-5e75e6fe6278/tmp_insert_all_1_1_0/\"] [\"/var/lib/clickhouse/store/b1a/b1a5f878-85cf-44da-bb9f-5e75e6fe6278/all_1_1_0/\"] in line 0: INSERT INTO schema_migrations (version, dirty, sequence) VALUES (1, 1, 1717801960995297780)","stacktrace":"main.main\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/migrate.go:128\nruntime.main\n\t/opt/hostedtoolcache/go/1.21.10/x64/src/runtime/proc.go:267"}