Hi team, I have just started trying out SigNoz and unable to send file logs from another host to SigNoz. Any pointers/direction appreciated! These are the steps I've tried so far SigNoz (v0.11.2) [192.168.200.20] SigNoz Setup: installed via ./install.sh SigNoz changes: removed hotrod from docker-compose NestJS app, dev OSX [192.168.200.187] NestJS using auto-node tracing SigNoz able to receive NestJS traces ✅ NestJS Logging / Winston Logs from backend.log, send to SigNoz Logs via otel-collector ❌ Example log entry in backend.log:

{"level":"info","message":"PassportModule dependencies initialized","service":"logger-service","timestamp":"2022-10-21T10:51:31.050Z"}

Following this doc: https://signoz.io/docs/userguide/collect_logs_from_file/ Docker run command:

docker run -d --name signoz-otel-collector --user root -v $(pwd)/backend.log:/tmp/backend.log:ro -v $(pwd)/signoz-winston-collector.yaml:/etc/otel/config.yaml signoz/signoz-otel-collector:0.55.3

signoz-winston-collector.yaml

receivers:
  filelog:
    include: [/tmp/backend.log]
    start_at: beginning
    operators:
      - type: json_parser
        id: nest-winston-json
        timestamp:
          parse_from: attributes.timestamp
          layout: '%Y-%m-%dT%H:%M:%S.%LZ'
      - type: move
        from: attributes.message
        to: body
      - type: remove
        field: attributes.timestamp
processors:
  batch:
    send_batch_size: 10 #low num for testing
    send_batch_max_size: 10 #low num for testing
    timeout: 10s
exporters:
  otlp/log:
    endpoint: <http://192.168.200.20:4317>
    tls:
      insecure: true
service:
  telemetry:
    logs:
      level: 'debug'
  pipelines:
    logs:
      receivers: [filelog]
      processors: [batch]
      exporters: [otlp/log]

Docker collector container logs

2022-10-22T02:31:52.529Z        info    service/telemetry.go:103        Setting up own telemetry...
2022-10-22T02:31:52.530Z        info    service/telemetry.go:138        Serving Prometheus metrics      {"address": ":8888", "level": "basic"}
2022-10-22T02:31:52.530Z        debug   pipelines/pipelines.go:343      Stability level {"kind": "exporter", "data_type": "logs", "name": "otlp/log", "stability": "beta"}
2022-10-22T02:31:52.530Z        debug   pipelines/pipelines.go:343      Stability level {"kind": "processor", "name": "batch", "pipeline": "logs", "stability": "stable"}
2022-10-22T02:31:52.530Z        debug   pipelines/pipelines.go:345      Stability level of component undefined  {"kind": "receiver", "name": "filelog", "pipeline": "logs", "stability": "undefined"}
2022-10-22T02:31:52.532Z        info    extensions/extensions.go:42     Starting extensions...
2022-10-22T02:31:52.532Z        info    pipelines/pipelines.go:74       Starting exporters...
2022-10-22T02:31:52.532Z        info    pipelines/pipelines.go:78       Exporter is starting... {"kind": "exporter", "data_type": "logs", "name": "otlp/log"}
2022-10-22T02:31:52.532Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] Channel created     {"grpc_log": true}
2022-10-22T02:31:52.532Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] original dial target is: "192.168.200.20:4317"      {"grpc_log": true}
2022-10-22T02:31:52.532Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] dial target "192.168.200.20:4317" parse failed: parse "192.168.200.20:4317": first path segment in URL cannot contain colon       {"grpc_log": true}
2022-10-22T02:31:52.532Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] fallback to scheme "passthrough"    {"grpc_log": true}
2022-10-22T02:31:52.532Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] parsed dial target is: {Scheme:passthrough Authority: Endpoint:192.168.200.20:4317 URL:{Scheme:passthrough Opaque: User: Host: Path:/192.168.200.20:4317 RawPath: ForceQuery:false RawQuery: Fragment: RawFragment:}}   {"grpc_log": true}
2022-10-22T02:31:52.532Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] Channel authority set to "192.168.200.20:4317"      {"grpc_log": true}
2022-10-22T02:31:52.532Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] Resolver state updated: {
  "Addresses": [
    {
      "Addr": "192.168.200.20:4317",
      "ServerName": "",
      "Attributes": null,
      "BalancerAttributes": null,
      "Type": 0,
      "Metadata": null
    }
  ],
  "ServiceConfig": null,
  "Attributes": null
} (resolver returned new addresses)     {"grpc_log": true}
2022-10-22T02:31:52.533Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] Channel switches to new LB policy "pick_first"      {"grpc_log": true}
2022-10-22T02:31:52.533Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1 SubChannel #2] Subchannel created    {"grpc_log": true}
2022-10-22T02:31:52.533Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1 SubChannel #2] Subchannel Connectivity change to CONNECTING{"grpc_log": true}
2022-10-22T02:31:52.533Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1 SubChannel #2] Subchannel picks a new address "192.168.200.20:4317" to connect     {"grpc_log": true}
2022-10-22T02:31:52.533Z        info    pipelines/pipelines.go:82       Exporter started.       {"kind": "exporter", "data_type": "logs", "name": "otlp/log"}
2022-10-22T02:31:52.533Z        info    pipelines/pipelines.go:86       Starting processors...
2022-10-22T02:31:52.533Z        info    pipelines/pipelines.go:90       Processor is starting...        {"kind": "processor", "name": "batch", "pipeline": "logs"}
2022-10-22T02:31:52.534Z        info    zapgrpc/zapgrpc.go:174  [core] pickfirstBalancer: UpdateSubConnState: 0xc0008d10d0, {CONNECTING <nil>}{"grpc_log": true}
2022-10-22T02:31:52.534Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] Channel Connectivity change to CONNECTING   {"grpc_log": true}
2022-10-22T02:31:52.534Z        info    pipelines/pipelines.go:94       Processor started.      {"kind": "processor", "name": "batch", "pipeline": "logs"}
2022-10-22T02:31:52.534Z        info    pipelines/pipelines.go:98       Starting receivers...
2022-10-22T02:31:52.534Z        info    pipelines/pipelines.go:102      Receiver is starting... {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-10-22T02:31:52.534Z        info    adapter/receiver.go:54  Starting stanza receiver        {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-10-22T02:31:52.534Z        debug   pipeline/directed.go:70 Starting operator       {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-10-22T02:31:52.534Z        debug   pipeline/directed.go:74 Started operator        {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-10-22T02:31:52.534Z        debug   pipeline/directed.go:70 Starting operator       {"kind": "receiver", "name": "filelog", "pipeline": "logs", "operator_id": "remove", "operator_type": "remove"}
2022-10-22T02:31:52.534Z        debug   pipeline/directed.go:74 Started operator        {"kind": "receiver", "name": "filelog", "pipeline": "logs", "operator_id": "remove", "operator_type": "remove"}
2022-10-22T02:31:52.534Z        debug   adapter/converter.go:143        Starting log converter  {"kind": "receiver", "name": "filelog", "pipeline": "logs", "worker_count": 1}
2022-10-22T02:31:52.534Z        info    pipelines/pipelines.go:106      Receiver started.       {"kind": "receiver", "name": "filelog", "pipeline": "logs"}
2022-10-22T02:31:52.535Z        info    service/collector.go:215        Starting signoz-otel-collector...       {"Version": "latest", "NumCPU": 6}
2022-10-22T02:31:52.535Z        info    service/collector.go:128        Everything is ready. Begin running and processing data.
2022-10-22T02:31:52.738Z        info    fileconsumer/file.go:178        Started watching file   {"kind": "receiver", "name": "filelog", "pipeline": "logs", "component": "fileconsumer", "path": "/tmp/backend.log"}
2022-10-22T02:31:52.744Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1 SubChannel #2] Subchannel Connectivity change to READY    {"grpc_log": true}
2022-10-22T02:31:52.744Z        info    zapgrpc/zapgrpc.go:174  [core] pickfirstBalancer: UpdateSubConnState: 0xc0008d10d0, {READY <nil>}    {"grpc_log": true}
2022-10-22T02:31:52.744Z        info    zapgrpc/zapgrpc.go:174  [core] [Channel #1] Channel Connectivity change to READY        {"grpc_log": true}

Tried the follow troubleshoot:

docker run -it --rm signoz/troubleshoot checkEndpoint --endpoint=192.168.200.20:4317
2022-10-22T01:40:57.763Z	INFO	troubleshoot/main.go:28	STARTING!
2022-10-22T01:40:57.764Z	INFO	checkEndpoint/checkEndpoint.go:41	checking reachability of SigNoz endpoint
2022-10-22T01:40:57.776Z	INFO	troubleshoot/main.go:46	Successfully sent sample data to signoz ...

@nitya-signoz may have more insights on this

@Pranay @nitya-signoz managed to get it to work. Updated the collector config to the following:

operators:
      - type: json_parser
        parse_to: body
      - type: time_parser
        parse_from: body.timestamp
        layout: '%Y-%m-%dT%H:%M:%S.%LZ'
      # - type: json_parser
      #   timestamp:
      #     parse_from: attributes.timestamp
      #     layout: '%Y-%m-%dT%H:%M:%S.%LZ'
      # - type: move
      #   from: attributes.message
      #   to: body
      - type: remove
        field: body.timestamp

Nice @Jason Loong 👌

Great let us know if you need any other help.