Hi! Running into an issue with pipelines. I'm trying to parse a json field

labels

from a log such as this one

{
  "body": "{\"raw_log\":\"{\\\"level\\\":\\\"info\\\",\\\"module\\\":\\\"server\\\",\\\"module\\\":\\\"txindex\\\",\\\"height\\\":28557212,\\\"time\\\":\\\"2024-09-12T16:13:47-04:00\\\",\\\"message\\\":\\\"indexed block events\\\"}\"}",
  "id": "2lz9RKpucUEwudqQjp7LieQ9U4W",
  "timestamp": 1726172028356,
  "attributes": {
    "com.hashicorp.nomad.alloc_id": "71f80e7a-31d8-9a51-d5c5-9ad19783d6a5",
    "container_name": "/chain-binary-71f80e7a-31d8-9a51-d5c5-9ad19783d6a5",
    "labels": "{\"com.hashicorp.nomad.alloc_id\":\"71f80e7a-31d8-9a51-d5c5-9ad19783d6a5\"}",
    "level": "info",
    "message": "indexed block events",
    "module": "txindex",
    "nomad_job_name": "testnet-validator",
    "time": "2024-09-12T16:13:47-04:00"
  },
  "resources": {},
  "severity_text": "",
  "severity_number": 0,
  "trace_id": "",
  "span_id": "",
  "trace_flags": 0
}

The preview in the frontend works as expected. When I save the pipeline, however, it does not work and I see these errors in the collector logs

2024-09-12T20:16:29.396Z	error	helper/transformer.go:102	Failed to process entry	{"kind": "processor", "name": "logstransform/pipeline_Test", "pipeline": "logs", "operator_id": "4c9ebbab-d8b1-4ecb-9e07-c42459db68ab", "operator_type": "json_parser", "error": "running if expr: interface conversion: interface {} is map[string]interface {}, not string (1:48)\n | attributes?.labels != nil && attributes.labels matches \"^\\\\s*{.*}\\\\s*$\"\n | ...............................................^", "action": "send"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*TransformerOperator).HandleEntryError|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*TransformerOperator).HandleEntryError>
	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.102.0/operator/helper/transformer.go:102
<http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWithCallback|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWithCallback>
	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.102.0/operator/helper/parser.go:105
<http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWith|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/helper.(*ParserOperator).ProcessWith>
	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.102.0/operator/helper/parser.go:98
<http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/parser/json.(*Parser).Process|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/parser/json.(*Parser).Process>
	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.102.0/operator/parser/json/parser.go:24
<http://github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/transformer/router.(*Transformer).Process|github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/operator/transformer/router.(*Transformer).Process>
	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.102.0/operator/transformer/router/transformer.go:57
<http://github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor.(*logsTransformProcessor).converterLoop|github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor.(*logsTransformProcessor).converterLoop>
	/home/runner/go/pkg/mod/github.com/open-telemetry/opentelemetry-collector-contrib/processor/logstransformprocessor@v0.102.0/processor.go:213

Any idea why this might be an issue? The pipeline executes the next step after the failed json parsing.

Hi @Mircea Colonescu 🙂 Are you extracting

labels

into attributes using a processor before the JSON parsing processor with errors? If yes, this error is happening because pipelines in the actual collector will see

attributes.labels

as an already parsed map/object and not a JSON string The preview works because the sample logs you see in the preview are fetched from the database (clickhouse) Since all attributes values get turned to strings before being stored in the database, the sample logs in the preview have attributes.labels as a string

Hi @Raj Kamal. Thanks for getting back to me. I am indeed using a JSON parser in the collector before anything else. So is there an easy way to parse this type of nested JSON? I'd like to extract all of the labels into their own fields, but I don't see any sort of loop functionality, and the number and names of the labels are not known beforehand.

There is no easy way to do it today but there is a workaround you can use. Please take a look at https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1725974230939089?thread_ts=1725907216.606139&cid=C01HWQ1R0BC for the workaround

Thank you. I'll try that and see if it works for me.