Hey team 👋 I'm running into an issue with the

k8s-infra

Helm chart. I’ve deployed SigNoz and the

k8s-infra

agent as a DaemonSet in my cluster, and everything seems to be running fine. However, when I open the SigNoz UI: • Under the Metrics tab, I’m getting: • "Error fetching metric details" • Under Infrastructure Monitoring, it shows: • "No host metrics data received yet. Infrastructure monitoring requires the OpenTelemetry system metrics. Please refer to this to learn how to send host metrics to SigNoz." Interestingly, when I create a dashboard provided by signoz, I can see data correctly. So it seems like the metrics are coming in, but not appearing under the Infra or Metrics tabs as expected. Could this be a configuration issue with the

k8s-infra

Helm values or something else? Would appreciate any guidance! 🙏

@Srikanth Chekuri ^

k8s-infra-otel-agent daemonset pod. 2025/04/21 024210 failed to upload metrics: Post "http://signoz-otel-collector.signoz.svc.cluster.local:4317/v1/metrics": net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00"

No host metrics data received yet. Infrastructure monitoring requires the OpenTelemetry system metrics. Please refer to this to learn how to send host metrics to SigNoz

This is not same as host metrics. What you see in k8s node metrics which is available under k8s tab

• “Error fetching metric details”

Please share the logs of query-service when this happens

k8s-infra-otel-agent daemonset pod.

2025/04/21 024210 failed to upload metrics: Post “http://signoz-otel-collector.signoz.svc.cluster.local:4317/v1/metrics”: net/http: HTTP/1.x transport connection broken: malformed HTTP response “\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00”

Please share more details. the port 4317 is for grpc, and the above endpoint in invalid.

i have fixed this issue, the isn't coming anymore & redeployed the k8s-infra. Even though no metrics are visible in infra-monitoring & metrics tabs

{"level":"info","ts":1745414082.5915973,"caller":"extensions/extensions.go:39","msg":"Starting extensions..."}
{"level":"info","ts":1745414082.5916293,"caller":"extensions/extensions.go:42","msg":"Extension is starting...","kind":"extension","name":"health_check"}
{"level":"info","ts":1745414082.5916557,"caller":"healthcheckextension@v0.109.0/healthcheckextension.go:33","msg":"Starting health_check extension","kind":"extension","name":"health_check","config":{"Endpoint":"0.0.0.0:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0,"Path":"/","ResponseBody":null,"CheckCollectorPipeline":{"Enabled":false,"Interval":"5m","ExporterFailureThreshold":5}}}
{"level":"info","ts":1745414082.5918877,"caller":"extensions/extensions.go:59","msg":"Extension started.","kind":"extension","name":"health_check"}
{"level":"info","ts":1745414082.5919077,"caller":"extensions/extensions.go:42","msg":"Extension is starting...","kind":"extension","name":"pprof"}
{"level":"info","ts":1745414082.592166,"caller":"pprofextension@v0.109.0/pprofextension.go:61","msg":"Starting net/http/pprof server","kind":"extension","name":"pprof","config":{"TCPAddr":{"Endpoint":"localhost:1777","DialerConfig":{"Timeout":0}},"BlockProfileFraction":0,"MutexProfileFraction":0,"SaveToFile":""}}
{"level":"info","ts":1745414082.592222,"caller":"extensions/extensions.go:59","msg":"Extension started.","kind":"extension","name":"pprof"}
{"level":"info","ts":1745414082.5922306,"caller":"extensions/extensions.go:42","msg":"Extension is starting...","kind":"extension","name":"zpages"}
{"level":"info","ts":1745414082.5922832,"caller":"zpagesextension@v0.109.0/zpagesextension.go:55","msg":"Registered zPages span processor on tracer provider","kind":"extension","name":"zpages"}
{"level":"info","ts":1745414082.5923383,"caller":"zpagesextension@v0.109.0/zpagesextension.go:65","msg":"Registered Host's zPages","kind":"extension","name":"zpages"}
{"level":"info","ts":1745414082.5924618,"caller":"zpagesextension@v0.109.0/zpagesextension.go:77","msg":"Starting zPages extension","kind":"extension","name":"zpages","config":{"Endpoint":"localhost:55679","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0}}
{"level":"info","ts":1745414082.5925102,"caller":"extensions/extensions.go:59","msg":"Extension started.","kind":"extension","name":"zpages"}
{"level":"info","ts":1745414082.592995,"caller":"kube/client.go:123","msg":"k8s filtering","kind":"processor","name":"k8sattributes","pipeline":"traces","labelSelector":"","fieldSelector":"spec.nodeName=gke-qa-cluster-cast-pool-cdbb3ab8"}
{"level":"info","ts":1745414082.5934746,"caller":"kube/client.go:123","msg":"k8s filtering","kind":"processor","name":"k8sattributes","pipeline":"metrics","labelSelector":"","fieldSelector":"spec.nodeName=gke-qa-cluster-cast-pool-cdbb3ab8"}
{"level":"info","ts":1745414082.5936198,"caller":"internal/resourcedetection.go:125","msg":"began detecting resource information","kind":"processor","name":"resourcedetection","pipeline":"metrics"}
{"level":"warn","ts":1745414084.5943499,"caller":"internal/resourcedetection.go:130","msg":"failed to detect resource","kind":"processor","name":"resourcedetection","pipeline":"metrics","error":"failed getting k8s node UID: failed to fetch node with name gke-qa-cluster-cast-pool-cdbb3ab8 from K8s API: Get \"<https://10.8.0.1:443/api/v1/nodes/gke-qa-cluster-cast-pool-cdbb3ab8>\": context deadline exceeded"}
{"level":"info","ts":1745414084.5946178,"caller":"internal/resourcedetection.go:139","msg":"detected resource information","kind":"processor","name":"resourcedetection","pipeline":"metrics","resource":{"cloud.account.id":"bikes-272910","cloud.availability_zone":"asia-south1-a","cloud.platform":"gcp_kubernetes_engine","cloud.provider":"gcp","host.id":"6498867631715134464","host.name":"gke-qa-cluster-cast-pool-cdbb3ab8","k8s.cluster.name":"qa-cluster","k8s.node.name":"gke-qa-cluster-cast-pool-cdbb3ab8","os.type":"linux","signoz.component":"otel-agent"}}
{"level":"info","ts":1745414084.5951672,"caller":"kube/client.go:123","msg":"k8s filtering","kind":"processor","name":"k8sattributes","pipeline":"logs","labelSelector":"","fieldSelector":"spec.nodeName=gke-qa-cluster-cast-pool-cdbb3ab8"}
{"level":"info","ts":1745414084.59528,"caller":"otlpreceiver@v0.109.0/otlp.go:103","msg":"Starting GRPC server","kind":"receiver","name":"otlp","data_type":"metrics","endpoint":"0.0.0.0:4317"}