Another question, I am trying to set up otel collector in docker-compose by following <https://signo...

09/16/2024, 12:24 AM

Another question, I am trying to set up otel collector in docker-compose by following https://signoz.io/opentelemetry/collector-nodejs/ but for a Golang Application. But I am getting

Copy code

otel-collector-1  | 2024-09-15T16:45:05.721-0700	error	scraperhelper/scrapercontroller.go:197	Error scraping metrics	{"kind": "receiver", "name": "hostmetrics", "data_type": "metrics", "error": "error reading username for process \"otelcol-contrib\" (pid 1): open /etc/passwd: no such file or directory", "scraper": "hostmetrics"}
otel-collector-1  | <http://go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport|go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport>
otel-collector-1  | 	<http://go.opentelemetry.io/collector/receiver@v0.109.0/scraperhelper/scrapercontroller.go:197|go.opentelemetry.io/collector/receiver@v0.109.0/scraperhelper/scrapercontroller.go:197>
otel-collector-1  | <http://go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1|go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1>
otel-collector-1  | 	<http://go.opentelemetry.io/collector/receiver@v0.109.0/scraperhelper/scrapercontroller.go:169|go.opentelemetry.io/collector/receiver@v0.109.0/scraperhelper/scrapercontroller.go:169>

I've uploaded the app in https://github.com/GaikwadPratik/signoz-test. Can someone please help me set up the app?

Srikanth Chekuri

09/16/2024, 3:25 AM

This error is independent of your application. It comes from hostmetrics and can safely be ignored.

09/16/2024, 3:58 AM

@Srikanth Chekuri thank you for replying. Any idea why logs from app container are not forwarded to signoz setup?

09/16/2024, 4:10 AM

I do have ports open and accessible

Copy code

PORT     STATE SERVICE
22/tcp   open  ssh
2181/tcp open  eforward
3301/tcp open  tarantool
4317/tcp open  opentelemetry
4318/tcp open  unknown
8123/tcp open  polipo
9000/tcp open  cslistener

09/16/2024, 5:50 AM

This is my docker compose

Copy code

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: app-container
    ports:
      - "5555:5555"
    environment:
      - OTEL_TRACES_EXPORTER=otlp
      - OTEL_EXPORTER_OTLP_ENDPOINT=otel-collector:4317
      - OTEL_NODE_RESOURCE_DETECTORS=env,host,os
      - OTEL_SERVICE_NAME=app-service
      - OTEL_EXPORTER_OTLP_INSECURE=true
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /etc/localzone:/etc/localzone:ro
    depends_on:
      - otel-collector
    networks:
      - app-network

  otel-collector:
    image: otel/opentelemetry-collector-contrib:latest
    command: ["--config", "/etc/otel-collector-config.yaml"]
    container_name: otel-collector-container
    volumes:
      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml:ro
      - /etc/localtime:/etc/localtime:ro
      - /etc/localzone:/etc/localzone:ro
    ports:
      - "4317:4317"
      - "4318:4318"
      - "8888:8888"
    environment:
      - OTEL_EXPORTER_OTLP_ENDPOINT=10.211.55.4:4318
      - OTEL_EXPORTER_OTLP_INSECURE=true
    networks:
      - app-network

networks:
  app-network:

09/16/2024, 6:00 AM

Any idea about

Copy code

app-container             | {"time":"2024-09-15T22:59:58-07:00","level":"INFO","msg":"traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: http2: frame too large\""}
app-container             | {"time":"2024-09-15T23:00:08-07:00","level":"INFO","msg":"traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: http2: frame too large\""}

09/16/2024, 6:20 AM

Furthermore, I ran the troubleshoot binary as well

Copy code

➤  ./troubleshoot checkEndpoint --endpoint=10.211.55.4:4317
2024-09-15T22:07:33.463-0700	INFO	workspace/main.go:28	STARTING!
2024-09-15T22:07:33.463-0700	INFO	checkEndpoint/checkEndpoint.go:41	checking reachability of SigNoz endpoint
2024-09-15T22:07:33.466-0700	INFO	workspace/main.go:46	Successfully sent sample data to signoz ...

09/16/2024, 6:44 AM

After removing

otel-collector

and it's references from docker-compose I am able to get traces to signoz but not logs. Any pointers would be appreciated

Srikanth Chekuri

09/17/2024, 5:16 AM

Share the otel-collector config

09/17/2024, 5:34 AM

@Srikanth Chekuri

Copy code

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
  # hostmetrics:
  #   collection_interval: 60s
  #   scrapers:
  #     cpu: {}
  #     disk: {}
  #     load: {}
  #     filesystem: {}
  #     memory: {}
  #     network: {}
  #     paging: {}
  #     process:
  #       mute_process_name_error: true
  #       mute_process_exe_error: true
  #       mute_process_io_error: true
  #     processes: {}
  prometheus:
    config:
      global:
        scrape_interval: 60s
      scrape_configs:
        - job_name: otel-collector-binary
          static_configs:
            - targets:
processors:
  batch:
    send_batch_size: 1000
    timeout: 10s
  resourcedetection:
    detectors: [env, system]
    timeout: 2s
    system:
      hostname_sources: [os]
extensions:
  health_check: {}
  zpages: {}
exporters:
  otlp:
    endpoint: 10.211.55.4:4317
    tls:
      insecure: true
  logging:
    verbosity: normal
service:
  telemetry:
    metrics:
      address: 0.0.0.0:8888
  extensions: [health_check, zpages]
  pipelines:
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    metrics/internal:
      receivers: [prometheus]
      processors: [resourcedetection, batch]
      exporters: [otlp]
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]

Srikanth Chekuri

09/17/2024, 6:34 AM

app-container | {"time":"2024-09-15T225958-07:00","level":"INFO","msg":"traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: http2: frame too large\""}

app-container | {"time":"2024-09-15T230008-07:00","level":"INFO","msg":"traces export: context deadline exceeded: rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: http2: frame too large\""}

What is the endpoint config used in the application

Vinod Ahlawat

09/17/2024, 1:30 PM

Did we get any fix for this issue?

09/17/2024, 5:09 PM

@Srikanth Chekuri https://github.com/GaikwadPratik/signoz-test/blob/58ecc10d7163b1c556b1576a6b961dd6fecac5b9/docker-compose.yml#L11

Srikanth Chekuri

09/17/2024, 5:27 PM

Which language/sdk does thie application use?

09/17/2024, 6:00 PM

GO. You should be able to access the repo on this commit.

156 Views

Open in Slack

Previous Next

SigNoz Community

SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.