Slackbot
09/01/2022, 1:31 PMPrashant Shahi
09/01/2022, 1:33 PMkubectl -n platform get pods -o=wide
Valentin Lorand
09/01/2022, 1:35 PMNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
chi-signoz-cluster-0-0-0 1/1 Running 0 14m 100.64.0.55 scw-k8s-vld-default-368a16ee0c6b403eb3ee422bcb <none> <none>
clickhouse-operator-6c966f59cd-dk7pv 2/2 Running 0 14m 100.64.0.197 scw-k8s-vld-default-368a16ee0c6b403eb3ee422bcb <none> <none>
my-release-signoz-alertmanager-0 0/1 Pending 0 14m <none> <none> <none> <none>
my-release-signoz-frontend-cf9b8d8c9-rkj7n 0/1 Init:0/1 0 14m 100.64.0.145 scw-k8s-vld-default-368a16ee0c6b403eb3ee422bcb <none> <none>
my-release-signoz-otel-collector-ddb76f957-kg7lk 0/1 CrashLoopBackOff 3 (26s ago) 74s 100.64.0.106 scw-k8s-vld-default-368a16ee0c6b403eb3ee422bcb <none> <none>
my-release-signoz-otel-collector-metrics-5dd74686c-9g9sc 0/1 CrashLoopBackOff 6 (2m55s ago) 14m 100.64.0.119 scw-k8s-vld-default-368a16ee0c6b403eb3ee422bcb <none> <none>
my-release-signoz-query-service-0 0/1 CrashLoopBackOff 7 (19s ago) 14m 100.64.0.98 scw-k8s-vld-default-368a16ee0c6b403eb3ee422bcb <none> <none>
my-release-zookeeper-0 1/1 Running 0 14m 100.64.0.19 scw-k8s-vld-default-368a16ee0c6b403eb3ee422bcb <none> <none>
Valentin Lorand
09/01/2022, 1:43 PM[chi-signoz-cluster-0-0-0] 2022.09.01 13:37:57.003959 [ 11 ] {} <Error> Access(user directories): from: ::ffff:100.64.0.119, user: admin: Authentication failed: Code: 195. DB::Exception: Connections from ::ffff:100.64.0.119 are not allowed. (IP_ADDRESS_NOT_ALLOWED), Stack trace (when copying this message, always include the lines below):
[chi-signoz-cluster-0-0-0] 2022.09.01 13:37:57.003780 [ 11 ] {} <Warning> AddressPatterns: Failed to check if the allowed client hosts contain address ::ffff:100.64.0.119. DB::Exception: Cannot getnameinfo(::ffff:100.64.0.119): Name or service not known, code = 198
Prashant Shahi
09/01/2022, 1:53 PM100.64.0.0/16
https://github.com/SigNoz/charts/blob/main/charts/signoz/values.yaml#L117-L120Prashant Shahi
09/01/2022, 1:54 PMclickhouse.allowedNetworkIps
listPrashant Shahi
09/01/2022, 1:54 PMValentin Lorand
09/01/2022, 2:24 PM[my-release-signoz-otel-collector-metrics-54d4778cbf-b2qs4 my-release-signoz-otel-collector-metrics-init] wget: bad address 'my-release-clickhouse:8123'
[my-release-signoz-otel-collector-metrics-54d4778cbf-b2qs4 my-release-signoz-otel-collector-metrics-init] waiting for clickhouseDB
Prashant Shahi
09/01/2022, 2:47 PM- "10.0.0.0/8"
- "172.16.0.0/12"
- "192.168.0.0/16"
Benjamin Carriou
09/01/2022, 2:50 PM[my-release-signoz-otel-collector-5cgbv my-release-signoz-otel-collector-init] waiting for clickhouseDB
[my-release-signoz-query-service-0 my-release-signoz-query-service-init] wget: bad address 'my-release-clickhouse:8123'
[my-release-signoz-query-service-0 my-release-signoz-query-service-init] waiting for clickhouseDB
[my-release-signoz-otel-collector-5cgbv my-release-signoz-otel-collector-init] wget: bad address 'my-release-clickhouse:8123'
[my-release-signoz-otel-collector-5cgbv my-release-signoz-otel-collector-init] waiting for clickhouseDB
[my-release-signoz-otel-collector-metrics-54d4778cbf-p6hp2 my-release-signoz-otel-collector-metrics-init] wget: bad address 'my-release-clickhouse:8123'
[my-release-signoz-otel-collector-metrics-54d4778cbf-p6hp2 my-release-signoz-otel-collector-metrics-init] waiting for clickhouseDB
[my-release-signoz-query-service-0 my-release-signoz-query-service-init] wget: bad address 'my-release-clickhouse:8123'
[my-release-signoz-query-service-0 my-release-signoz-query-service-init] waiting for clickhouseDB
Benjamin Carriou
09/01/2022, 2:51 PMBenjamin Carriou
09/01/2022, 2:52 PMI0901 14:48:48.489450 1 labeler.go:80] OPERATOR_POD_NAMESPACE=platform OPERATOR_POD_NAME=my-release-clickhouse-operator-5cbd5c88d5-6frhb
E0901 14:48:48.576474 1 labeler.go:209] labelDeployment():platform/my-release-clickhouse-operator:ERROR get Deployment platform/my-release-clickhouse-operator
E0901 14:48:48.576707 1 controller.go:488] Run():ERROR label objects, will retry. Err: deployments.apps "my-release-clickhouse-operator" is forbidden: User "system:serviceaccount:platform:my-release-clickhouse-operator" cannot get resource "deployments" in API group "apps" in the namespace "platform"
I0901 14:48:53.577790 1 labeler.go:80] OPERATOR_POD_NAMESPACE=platform OPERATOR_POD_NAME=my-release-clickhouse-operator-5cbd5c88d5-6frhb
E0901 14:48:53.688019 1 labeler.go:209] labelDeployment():platform/my-release-clickhouse-operator:ERROR get Deployment platform/my-release-clickhouse-operator
E0901 14:48:53.688047 1 controller.go:488] Run():ERROR label objects, will retry. Err: deployments.apps "my-release-clickhouse-operator" is forbidden: User "system:serviceaccount:platform:my-release-clickhouse-operator" cannot get resource "deployments" in API group "apps" in the namespace "platform"
Prashant Shahi
09/02/2022, 2:02 AMPrashant Shahi
09/02/2022, 2:08 AMPrashant Shahi
09/02/2022, 2:18 AMPrashant Shahi
09/02/2022, 2:19 AMoverride-values.yaml
, do share that as well.Prashant Shahi
09/02/2022, 3:41 AMdeployments.apps "my-release-clickhouse-operator" is forbiddenI was able to reproduce the error above. Creating PR to fix it in some time.
Benjamin Carriou
09/02/2022, 7:27 AMPrashant Shahi
09/02/2022, 8:00 PMPrashant Shahi
09/05/2022, 9:33 AM