hi ! I'm trying to export k8s infra metrics signo...
# support
j
hi ! I'm trying to export k8s infra metrics signoz as indicated here: https://signoz.io/docs/tutorial/kubernetes-infra-metrics/#steps-to-export-k8s-metrics-to-signoz In my case I'm running Signoz in the same cluster where application are, therefore updated the endpoint accordingly but I can't plot any metrics as indicated in your docs. One note is that I have installed the collector through Otel operator, and I had it up and running. So I expected updating the endpoint should be enough ? So to make it clear I have the following
Copy code
āžœ  ~ k -n platform get pod
NAME                                                        READY   STATUS    RESTARTS   AGE
chi-signoz-cluster-0-0-0                                    1/1     Running   0          19h
clickhouse-operator-8cff468-jnhwn                           2/2     Running   0          19h
my-release-signoz-alertmanager-0                            1/1     Running   0          19h
my-release-signoz-frontend-679745cc99-4xrwj                 1/1     Running   0          19h
my-release-signoz-otel-collector-54d458dd8b-z24bp           1/1     Running   0          19h
my-release-signoz-otel-collector-metrics-5dc97557f6-fngtt   1/1     Running   0          19h
my-release-signoz-query-service-0                           1/1     Running   0          19h
my-release-zookeeper-0                                      1/1     Running   0          19h

āžœ  ~ k -n testns-0 get pod
NAME                        READY   STATUS    RESTARTS   AGE
daemonset-collector-8xm57   1/1     Running   0          19h
daemonset-collector-kwbpn   1/1     Running   0          19h
daemonset-collector-p9bcv   1/1     Running   0          19h
and trying to get the receiver/exporter configured in
testns-0
to export metrics to Signoz what could I be missing ? Created issue here https://github.com/SigNoz/signoz/issues/1011 Thank you,
p
@Prashant Shahi Would you have more insights on this?
hey @Jonathan Berrios thanks for sharing the issue. Seems like you are already getting some help on this in the github issue conversation - https://github.com/SigNoz/signoz/issues/1011 Lets chat more there
šŸ‘ 1
j
@Prashant Shahi hi here ! can we follow up here please. I added the
kubeletstats
but still doesn't do.
p
@Jonathan Berrios if possible can you please share the otelCollector config?
Have you included
kubeletstats
in the pipeline? It is either that or the SigNoz endpoint is inaccurate or inaccessible.
j
@Prashant Shahi hi ! here it, yes I included it:
Copy code
apiVersion: <http://opentelemetry.io/v1alpha1|opentelemetry.io/v1alpha1>
kind: OpenTelemetryCollector
metadata:
  name: daemonset
  namespace: opentelemetry-collector
spec:
  mode: daemonset
  #hostNetwork: true
  image: otel/opentelemetry-collector-contrib:0.43.0
  securityContext:
    privileged: true
  env:
    - name: K8S_NODE_NAME
      valueFrom:
        fieldRef:
          fieldPath: spec.nodeName
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
          http:

      # Data sources: metrics
      prometheus:
        config:
          scrape_configs:
            - job_name: "otel-collector"
              scrape_interval: 5s
              static_configs:
                - targets: ["localhost:8888"]

      # Data sources: metrics
      hostmetrics:
        scrapers:
          cpu:
          disk:
          filesystem:
          load:
          memory:
          network:
      kubeletstats:
        collection_interval: 60s
        auth_type: "serviceAccount"
        endpoint: "$${K8S_NODE_NAME}:10250"
        insecure_skip_verify: true

    exporters:
      prometheus:
        endpoint: "latest-prometheus-server.default.svc.cluster.local:80"
        namespace: "default"

      otlp:
        endpoint: "my-release-signoz-otel-collector.platform.svc.cluster.local:4317"
        tls:
          insecure: true

    service:
      pipelines:
        metrics:
          receivers: [otlp,hostmetrics,kubeletstats,prometheus]
          processors: []
          exporters: [prometheusremotewrite,otlp]
I should mention, that previous to this, I found out that there wasn't a cluster role neither a cluster role binding for the collector as I installed from the operator CRD. So I worked around that by creating a cluster role binding with wide permissions, for testing only for now until I can get this to work. Please let me know any thoughts you may have. Very appreciated,
I was able to solve this šŸ™‚
šŸŽ‰ 1
p
@Jonathan Berrios That's great to hear! my bad for missing out on the thread discussion.. We are working on the next major releases.