https://signoz.io logo
Join Slack
Powered by
This message was deleted.
# support
s
This message was deleted.
s
wget: bad address 'signoz-clickhouse:8123' Mon, Jan 15 2024 111758 pm waiting for clickhouseDB
Here is the error I see on the schema migrator init container. can someone help here wget: bad address 'signoz-clickhouse:8123' Mon, Jan 15 2024 111758 pm waiting for clickhouseDB
s
The migrator doesn't use http. Please use tcp port 9000. Example: 
<tcp://localhost:9000>
p
can you check if the clickhouse cluster is available and ready? That error usually happens when the CHI pods are not there or the clickhouse cluster is no healthy.
s
Clickhouse Cluster is in ready state. Schema migrator is trying to connect to bad address 'signoz-clickhouse:8123'. Where do I need to change this in yaml
p
Unless you changed it in override config, the default values should work. Can you share the output of the following?
Copy code
kubectl get pods -n platform
Also, do you only have that 
waiting for clickhouseDB
in migrator pod or any other logs as well?
s
This is only throwing error with bad address on migrator pod
s
I am running into the same issue when trying to bring up signoz. @Suvrat were you able to resolve it? If so, how?
I have components are stuck in Init
Copy code
my-release-signoz-alertmanager-0                            0/1     Init:0/1   0          39m
my-release-signoz-frontend-6c9d9f8898-687v2                 0/1     Init:0/1   0          39m
my-release-signoz-otel-collector-67996c94f9-7p98f           0/1     Init:0/1   0          39m
my-release-signoz-otel-collector-metrics-686fb5dfdx-gmbl9   0/1     Init:0/1   0          39m
my-release-signoz-query-service-0                           0/1     Init:0/1   0          39m
my-release-signoz-schema-migrator-init-qfcr3                0/1     Init:0/2   0          39m
p
When this happened to me it was because they were waiting on the specific schema migrator job, and that was due to the migrator requiring a function that only existed on a latter version of clickhouse.
s
What version of ClickHouse are you using? Please use the 24.1.2+
p
@Srivatsan Sundararajan I see that CHI pod missing. This usually happens due to unclean deletion of previous ClickHouse cluster.
Other pods depend on the ClickHouse pod to be ready, hence they are stuck in init phase
@Srivatsan Sundararajan You could either try with 
helm upgrade ...
first, if that doesn't work cleanup previously installed set up and re-install. Instructions to clean deletion: https://signoz.io/docs/operate/kubernetes/#uninstall-signoz-cluster
👀 1
s
Tried re-installing, no luck
Copy code
chi-my-release-clickhouse-cluster-0-0-0                     0/1     Init:CrashLoopBackOff   8 (47s ago)   36m
my-release-clickhouse-operator-6485b98d64-h726j             2/2     Running                 2 (36m ago)   36m
my-release-k8s-infra-otel-agent-6q5zb                       0/1     Pending                 0             36m
my-release-k8s-infra-otel-agent-7fbpb                       1/1     Running                 0             36m
my-release-k8s-infra-otel-agent-fsdxg                       1/1     Running                 0             36m
my-release-k8s-infra-otel-agent-mjkxm                       1/1     Running                 0             36m
my-release-k8s-infra-otel-agent-s6v2c                       1/1     Running                 0             36m
my-release-k8s-infra-otel-agent-s7dxb                       0/1     Pending                 0             36m
my-release-k8s-infra-otel-deployment-7859f4b8c8-lq247       1/1     Running                 0             36m
my-release-signoz-alertmanager-0                            0/1     Init:0/1                0             36m
my-release-signoz-frontend-84b8d89c86-2k6tt                 0/1     Init:0/1                0             36m
my-release-signoz-otel-collector-64fc65c9c6-d64r9           0/1     Init:0/1                0             36m
my-release-signoz-otel-collector-metrics-77c5b54cfd-8f89r   0/1     Init:0/1                0             36m
my-release-signoz-query-service-0                           0/1     Init:0/1                0             36m
my-release-signoz-schema-migrator-init-g8sfd                0/1     Init:0/2                0             36m
my-release-zookeeper-0                                      1/1     Running                 0             36m
Container logs for 
chi-my-release-clickhouse-cluster-0-0-0
Copy code
+ wget -O /tmp/histogramQuantile <https://github.com/SigNoz/signoz/raw/develop/deploy/docker/clickhouse-setup/user_scripts/histogramQuantile>
Connecting to <http://github.com|github.com> (192.30.255.113:443)
wget: can't connect to remote host (192.30.255.113): Operation timed out
+ mv /tmp/histogramQuantile /var/lib/clickhouse/user_scripts/histogramQuantile
mv: can't rename '/tmp/histogramQuantile': No such file or directory
+ chmod +x /var/lib/clickhouse/user_scripts/histogramQuantile
chmod: /var/lib/clickhouse/user_scripts/histogramQuantile: No such file or directory
s
We use UDF to calculate the Prometheus histogram quantile and it requires access to GitHub. Please make sure it is reachable.
s
When I login to the container and run the wget it works fine
but the clickhouse cluster init is failing
s
Does it consistently happen?
s
Yes. I've not yet been able to bring up the cluster until now.
35 Views
Open in Slack
PreviousNext
Powered by