https://signoz.io
Join Slack
This message was deleted.
# support
s
This message was deleted.
p
That’s strange. ClickHouse in cluster mode requires zookeeper pod to be ready. Can you share 
kubectl pod describe
for the zookeeper pod? Also, make sure you have sufficient resources in the cluster/machine.
👍 1
s
kubectl describe my-release-zookeeper-0 pod error: the server doesn't have a resource type "my-release-zookeeper-0"
p
complete command:
Copy code
kubectl -n platform describe pod/my-release-zookeeper-0
s
Warning FailedScheduling 4m40s (x228 over 4h12m) default-scheduler 0/4 nodes are available: 4 node(s) had volume node affinity conflict.
but it works properly till last week
why suddenly get this error
p
volume node affinity conflict
error happens when the PVC used by the pod is scheduled on different zones/region.
It is an issue seen in kubernetes cluster with nodes from multiple zones/regions. I had encountered this before as well. I resolved it by setting 
nodeAffinity
for those components to match the same zone of the PVC.
👍 1
if you do not care about data loss, you can go about deleting the PVC and restart statefulset pod(s). It should spawn the new PVCs in same zone as that of the pods.
s
No, i need data
@Prashant Shahi, Since zookeeper, alert manager and query service pvc where facing this issue, i won't face any metric data loss after deleting the affected pvc. thanks
p
Since zookeeper, alert manager and query service pvc where facing this issue, i won't face any metric data loss after deleting the affected pvc.
actually SQLite would be removed since it is attached to 
query-service
statefulset.
That would mean data related to user credentials, alerts, and dashboards would be affected.
s
Yes i was supposed to sign up again
And recreated the dashboard using config json
p
yes, that is correct
btw, which cloud vendor are you using for managing K8s cluster?
s
In what frequency do you collect metrics and persist to db
Aws
p
In what frequency do you collect metrics and persist to db
depends on your collection interval. By default it should be 30s for Hostmetrics and K8s Metrics.
17 Views
Open in Slack
PreviousNext
Powered by

SigNoz Community

SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.

Powered by