Hey! So I upgraded to latest version with helm did...
# support
v
Hey! So I upgraded to latest version with helm did the CRD updates. Everything started , but i just dont seem to get data about k8s_node_cpu_utilization or any other, however container metrics work just fine . Any ideas how to approach it ?
what makes things super strange that it somehow does not retrieve metrics even from past 😞 so dashboards and alerts are somewhat useless
with further testing seems that queries without variables & aggregrations return data while all aggregations seems to fail when using variables in query builder
p
@Srikanth Chekuri did anything change in the aggregations recently?
v
for more reference im using https://github.com/SigNoz/dashboards/blob/main/k8s-infra-metrics/kubernetes-metrics.json . And I have two clusters under Signoz monitoring , i think i might be a factor here cause with refreshes sometimes the metric apprears to display value
and issue seems only occur with Value type of visualization ,
s
@Prashant Shahi I didn’t understand your question. We have not changed anything in aggregation. It might be an issue with json and vars.
v
is there anything i could help with ? getting some logs/data so it would be easier to reproduce ?
p
@Viljar Võidula OtelAgent of K8sInfra chart is responsible for collecting kubelet stats metrics. You could try restarting the collector in the affected K8s clusters.
Copy code
kubectl rollout restart -n platform daemonset -l=<http://app.kubernetes.io/component=otel-agent|app.kubernetes.io/component=otel-agent>
v
okey i give it a go , i intially just restarted pods after deployment
but later - im currently out of office
p
I think I am able to reproduce your issue with number widget and query-builder.
Let me share my finding with my team and will get to you later.
v
thank you so much
p