Hello i m using fluent bit to send logs to signoz The log is SigNoz Community #support

Hello, i'm using fluent-bit to send logs to signoz...

Ulisses Silva

09/21/2023, 6:50 PM

Hello, i'm using fluent-bit to send logs to signoz. The log is in the json format and this json contains an attribute named "other_info" that contains an escaped json string. I need to convert this "other_info" attribute to appear structured in the SigNoz log viewer. Where else should I configure besides fluentbit? log example:

Copy code

{
  "@timestamp": "2023-09-21T15:38:31.950Z",
  "log.level": "INFO",
  "message": "OK GET /mnt/index.do",
  "ecs.version": "1.2.0",
  "service.name": "the_app",
  "event.dataset": "the_app",
  "process.thread.name": "http-nio-8080-exec-4",
  "log.logger": "the_app.logger",
  "app_module": "app_module",
  "duration": "1442235497",
  "http_request_method": "GET",
  "http_url_path": "/mnt/index.do",
  "user_id": "123456",
  "span_id": "b6f1a671a3b298fe",
  "trace_flags": "01",
  "trace_id": "38a8eb9837cce5bb0e7f9276018b1985",
  "other_info": "{\"a\":\"123\",\"b\":\"456\",\"c\":\"789\",\"d\":\"987\",\"e\":\"654\",\"f\":\"321\"}"
}

fluent-bit config:

Copy code

[INPUT]
    Name    tail
    Tag     log4j2.ecs.*
    Path    /path/to/logs/app-json.log
    Parser  log4j2.ecs.json.parser

[OUTPUT]
    Name          forward
    Match         *
    Host          localhost
    Port          24224

parser config

Copy code

[PARSER]
    Name   log4j2.ecs.json.parser
    Format json
    Time_Key @timestamp
    Time_Format %Y-%m-%dT%H:%M:%S.%LZ
    Decode_Field_As    escaped     other_info do_next
    Decode_Field_As    json        other_info

Prashant Shahi

09/21/2023, 7:06 PM

cc @nitya-signoz

nitya-signoz

09/22/2023, 4:42 AM

Please use the json parser operator https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/docs/operators/json_parser.md

Ulisses Silva

09/22/2023, 12:25 PM

The config bellow results in error:

Copy code

processors:
  logstransform:
    operators:
      - type: trace_parser
        trace_id:
          parse_from: attributes.trace_id
        span_id:
          parse_from: attributes.span_id
      - type: json_parser
        parse_from: other_info
      - type: remove
        field: attributes.trace_id
      - type: remove
        field: attributes.span_id

Error:

Copy code

signoz-otel-collector          | Error: failed to get config: cannot unmarshal the configuration: 1 error(s) decoding:
signoz-otel-collector          | 
signoz-otel-collector          | * error decoding 'processors': error reading configuration for "logstransform": 1 error(s) decoding:
signoz-otel-collector          | 
signoz-otel-collector          | * error decoding 'operators[1]': unmarshal to json_parser: 1 error(s) decoding:
signoz-otel-collector          | 
signoz-otel-collector          | * error decoding 'parse_from': unrecognized prefix

Ulisses Silva

09/22/2023, 12:36 PM

I changed the app logger to log the field "other_info" as a json instead of a formatted json string and removed the json_parser. The field is show in the logs explorer as a json formated string: other_info [Type:Tag] [Data type: String] "{\"a\":\"123\",\"b\":\"456\",\"c\":\"789\",\"d\":\"987\",\"e\":\"654\",\"f\":\"321\"}" Added the json_parser againg to the logstransformer:

Copy code

- type: json_parser
  parse_from: attributes.other_info

Results in:

Copy code

signoz-otel-collector | 2023-09-22T12:32:36.175Z	error	helper/transformer.go:99	Failed to process entry
{
  "kind": "processor",
  "name": "logstransform",
  "pipeline": "logs",
  "operator_id": "json_parser",
  "operator_type": "json_parser",
  "error": "type map[string]interface {} cannot be parsed as JSON",
  "action": "send",
  "entry": {
...

Ulisses Silva

09/22/2023, 12:49 PM

full error from signoz-otel-collector container:

Copy code

signoz-otel-collector          | 2023-09-22T12:43:01.273Z	error	helper/transformer.go:99	Failed to process entry	{"kind": "processor", "name": "logstransform", "pipeline": "logs", "operator_id": "json_parser", "operator_type": "json_parser", "error": "type map[string]interface {} cannot be parsed as JSON", "action": "send", "entry": {"observed_timestamp": "2023-09-22T12:43:01.173937618Z", "timestamp": "2023-09-22T12:43:01.087Z", "body": "ERROR POST /some/url/test", "attributes": {"app_module": "module_a", "duration": "10222951", "ecs.version": "1.2.0", "error_message": "Some error message", "error_type": "some.java.ExceptionClass", "event.dataset": "the_app", "fluent.tag": "...", "other_info": {"a": "123", "b": "456", "c": "789", "d": "987", "e": "654", "f": "321"}, "http_request_method": "POST", "http_url_path": "/some/url/test", "log.level": "INFO", "log.logger": "json.logger", "process.thread.name": "http-nio-8080-exec-3", "service.name": "the_app", "status_code": "ERROR", "trace_flags": "01", "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/116.0.0.0 Safari/537.36", "user_id": "kj4l5j23lç4j62jlkj5lk", "user_ip_address": "0:0:0:0:0:0:0:1", "uuid": "a66e242b-95a3-484b-b9c8-8d730289baf6"}, "severity": 0, "scope_name": ""}}
signoz-otel-collector          | <http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*TransformerOperator).HandleEntryError|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*TransformerOperator).HandleEntryError>
signoz-otel-collector          | 	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.79.0/operator/helper/transformer.go:99
signoz-otel-collector          | <http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ParseWith|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ParseWith>
signoz-otel-collector          | 	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.79.0/operator/helper/parser.go:140
signoz-otel-collector          | <http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWithCallback|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWithCallback>
signoz-otel-collector          | 	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.79.0/operator/helper/parser.go:112
signoz-otel-collector          | <http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWith|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWith>
signoz-otel-collector          | 	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.79.0/operator/helper/parser.go:98
signoz-otel-collector          | <http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/parser/json.(*Parser).Process|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/parser/json.(*Parser).Process>
signoz-otel-collector          | 	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.79.0/operator/parser/json/json.go:62
signoz-otel-collector          | <http://github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor.(*logsTransformProcessor).converterLoop|github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor.(*logsTransformProcessor).converterLoop>
signoz-otel-collector          | 	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor@v0.79.0/processor.go:143
signoz-otel-collector          | 2023-09-22T12:43:01.273Z	error	logstransformprocessor@v0.79.0/processor.go:144	processor encountered an issue with the pipeline	{"kind": "processor", "name": "logstransform", "pipeline": "logs", "error": "type map[string]interface {} cannot be parsed as JSON"}
signoz-otel-collector          | <http://github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor.(*logsTransformProcessor).converterLoop|github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor.(*logsTransformProcessor).converterLoop>
signoz-otel-collector          | 	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor@v0.79.0/processor.go:144

nitya-signoz

09/23/2023, 4:39 AM

You will have to pass the entire body to json parser and the move it, here is an example https://github.com/SigNoz/logs-benchmark/blob/0b2451e6108d8fa5fdd5808c4e174bd52b9d55d3/signoz/signoz-client/otel-collector-config.yaml#L12

2 Views

Open in Slack

Previous Next