Hey team. We're in the process of evaluating SigNoz. I was attempting to get it spun up on a Talos K8 cluster, but am not getting any kubernetes/pods metrics. Just installing a fresh

signoz/signoz

at

0.50.0

helm chart with default values sets everything up properly, no errors, but I am not getting any Kubernetes metrics out of the box. I read the documentation at https://signoz.io/docs/install/kubernetes/others/ and it stated the k8s-infra chart should be included, but as of https://github.com/SigNoz/charts/commit/a99b93fbc32801eca528e9046eddcc68a625d5ae, it appeared to be removed as a dependency. I deployed the

k8s-infra

helm chart with default values and still no cigar. I don't see any errors on any of these pods logs, storage is provisioned correctly. It appears

k8s-infra

is collecting metrics looking at the logs. Took a look at the support channel and found https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1723457860195989. I confirmed that if I delete the collector service, I am getting a TCP dial failure in the agent logs for the insecure port, so the configuration seems to be correct for this case. Looking for a gentle nudge in the right direction if possible, thanks for your time!

What do logs of signoz-otel-agent and otel-deployment show?

agent

Please share the configmap of these resources. I think only logging (usually used for troubleshooting) is enabled but not otlp.

Thank you!

Please use

0.11.10

k8s-infra version

I'm still having the same issue on that version too. I got an error stating that an endpoint must be set for OTEL Collection. I set it, errors went away and still not getting metrics/logs forwarded through. Here is a helmfile for reproduction/context

repositories:
  - name: signoz
    url: <https://charts.signoz.io>

releases:
  - name: subsys-signoz
    namespace: subsys-signoz
    chart: signoz/signoz
    version: 0.50.0
    disableValidation: true
    values:
      - global:
          clusterName: c1
          deploymentEnvironment: prod
  - name: subsys-signoz-infra
    namespace: subsys-signoz
    chart: signoz/k8s-infra
    version: 0.11.10
    values:
      - global:
          cloud: others
          clusterName: c1
          deploymentEnvironment: prod
      - otelCollectorEndpoint: subsys-signoz-otel-collector:4317

Please share the logs and configmap of agent

agent

This looks good and you should see the data. Can you share the more logs of clickhouse?

full-clickhouse

What are the memory resources given to SigNoz?

Memory is whatever the chart requests, parent machines have 128gb of RAM and there is plenty available Yes on QEMU. Is there something special I must do for this setup? I just have these on Proxmox machines. I wanted to do an evaluation with my existing hardware prior to promoting to a cloud environment.

Create a new file and set in

users.d

settings folder

<allow_simdjson>0</allow_simdjson>

<profiles>
        <default>
            <allow_simdjson>0</allow_simdjson>
        </default>
    </profiles>