Hi, I have logs in multiple directories one for ea...
# support
b
Hi, I have logs in multiple directories one for each microservice. I have configured filelog/auth-service, filelog/documents-service ... etc in otel-collector but I see only auth-service logs in logs-explorer. What am I missing? Here is my otel-collector-config.yaml. receivers: hostmetrics: collection_interval: 10s scrapers: cpu: {} memory: {} disk: {} filesystem: {} load: {} network: {} paging: {} processes: {} nginx: endpoint: "http://api.buho.com.do/status" collection_interval: 10s filelog/nginx: include: [/var/log/nginx/*.log] otlp: protocols: http: {} filelog/auth-service: include: [/logs/auth-service/*.log] start_at: beginning poll_interval: 1000ms operators: - type: json_parser id: parse_auth_logs parse_from: body timestamp: parse_from: attributes.timestamp layout_type: strptime layout: '%Y-%m-%d %H:%M:%S' severity: parse_from: attributes.level filelog/jobs-service: include: [/logs/jobs-service/*.log] start_at: beginning poll_interval: 1000ms operators: - type: json_parser id: parse_jobs_service_logs parse_from: body timestamp: parse_from: attributes.timestamp layout_type: strptime layout: '%Y-%m-%d %H:%M:%S' parse_to: attributes.timestamp severity: parse_from: attributes.level parse_to: attributes.severity filelog/tasks-service: include: [/logs/tasks-service/*.log] start_at: beginning poll_interval: 1000ms operators: - type: json_parser id: parse_tasks_service_logs parse_from: body timestamp: parse_from: attributes.timestamp layout_type: strptime layout: '%Y-%m-%d %H:%M:%S' parse_to: attributes.timestamp severity: parse_from: attributes.level parse_to: attributes.severity prometheus: config: global: scrape_interval: 60s scrape_configs: - job_name: otel-collector static_configs: - targets: - localhost:8888 labels: job_name: otel-collector processors: batch: send_batch_size: 10000 send_batch_max_size: 11000 timeout: 10s attributes: actions: - key: app value: nginx action: insert resourcedetection: # Using OTEL_RESOURCE_ATTRIBUTES envvar, env detector adds custom labels. detectors: [env, system] timeout: 2s signozspanmetrics/delta: metrics_exporter: clickhousemetricswrite, signozclickhousemetrics metrics_flush_interval: 60s latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 50ms, 100ms, 250ms, 500ms, 1000ms, 1400ms, 2000ms, 5s, 10s, 20s, 40s, 60s ] dimensions_cache_size: 100000 aggregation_temporality: AGGREGATION_TEMPORALITY_DELTA enable_exp_histogram: true dimensions: - name: service.namespace default: default - name: deployment.environment default: default # This is added to ensure the uniqueness of the timeseries # Otherwise, identical timeseries produced by multiple replicas of # collectors result in incorrect APM metrics - name: signoz.collector.id - name: service.version - name: browser.platform - name: browser.mobile - name: k8s.cluster.name - name: k8s.node.name - name: k8s.namespace.name - name: host.name - name: host.type - name: container.name extensions: health_check: endpoint: 0.0.0.0:13133 pprof: endpoint: 0.0.0.0:1777 exporters: otlp/log: endpoint: 0.0.0.0:4318 tls: insecure: true otlphttp: endpoint: 0.0.0.0:4318 tls: insecure: true clickhousetraces: datasource: tcp://clickhouse:9000/signoz_traces low_cardinal_exception_grouping: ${env:LOW_CARDINAL_EXCEPTION_GROUPING} use_new_schema: true clickhousemetricswrite: endpoint: tcp://clickhouse:9000/signoz_metrics disable_v2: true resource_to_telemetry_conversion: enabled: true clickhousemetricswrite/prometheus: endpoint: tcp://clickhouse:9000/signoz_metrics disable_v2: true signozclickhousemetrics: dsn: tcp://clickhouse:9000/signoz_metrics clickhouselogsexporter: dsn: tcp://clickhouse:9000/signoz_logs timeout: 10s use_new_schema: true # debug: {} service: telemetry: logs: encoding: json metrics: address: 0.0.0.0:8888 extensions: - health_check - pprof pipelines: traces: receivers: [otlp] processors: [signozspanmetrics/delta, batch] exporters: [clickhousetraces] metrics: receivers: [otlp, hostmetrics, nginx] processors: [batch] exporters: [clickhousemetricswrite, signozclickhousemetrics] metrics/prometheus: receivers: [prometheus] processors: [batch] exporters: [clickhousemetricswrite/prometheus, signozclickhousemetrics] logs: receivers: [otlp, filelog/auth-service, filelog/jobs-service, filelog/tasks-service] processors: [batch] exporters: [clickhouselogsexporter]