Hey Guys, I've installed k8s-infra using helm v0.11.18, I'm seeing for all my workloads severity_text is coming as empty string. Is there a config that I'm missing?

Please use pipelines to parse severity_text https://signoz.io/docs/logs-pipelines/introduction/ . https://signoz.io/docs/logs-pipelines/processors/#severity-parser

@nitya-signoz One more doubt, whenever I try to run IN filter in logs explorer from body, I get the below issue, any ideas on this how to resolve it.

Did you use the pipeline preview feature to test out the changes ?

Payload:

{
  "start": 1731553526000,
  "end": 1731575126000,
  "step": 60,
  "variables": {},
  "compositeQuery": {
    "queryType": "builder",
    "panelType": "graph",
    "fillGaps": false,
    "builderQueries": {
      "A": {
        "dataSource": "logs",
        "queryName": "A",
        "aggregateOperator": "count",
        "aggregateAttribute": {
          "id": "------false",
          "dataType": "",
          "key": "",
          "isColumn": false,
          "type": "",
          "isJSON": false
        },
        "timeAggregation": "rate",
        "spaceAggregation": "sum",
        "functions": [],
        "filters": {
          "op": "AND",
          "items": [
            {
              "id": "8c230243",
              "key": {
                "key": "body.level",
                "dataType": "",
                "type": "",
                "isColumn": false,
                "isJSON": false
              },
              "op": "in",
              "value": [
                "error",
                "info"
              ]
            }
          ]
        },
        "expression": "A",
        "disabled": false,
        "stepInterval": 60,
        "having": [],
        "limit": null,
        "orderBy": [
          {
            "columnName": "timestamp",
            "order": "desc"
          }
        ],
        "groupBy": [
          {
            "key": "severity_text",
            "dataType": "string",
            "type": "",
            "isColumn": true,
            "isJSON": false,
            "id": "severity_text--string----true"
          }
        ],
        "legend": "",
        "reduceTo": "avg"
      }
    }
  }
}

Response :

{
  "status": "error",
  "data": {},
  "errorType": "internal",
  "error": "error in builder queries"
}

https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1731574955860999?thread_ts=1731391679.001799&cid=C01HWQ1R0BC

Is your log body a json ? if yes you will have to use json parser as the first processor and then you can parse the severity later. check out this doc https://signoz.io/docs/logs-pipelines/guides/json/

Ahh that might be it

Let me know how it goes.

And do I need to do that for the above query issue as well? where IN is not working?

I will need more info in what your log structure is. First try to use the json parse and get the parsing done then we can look into the other.

Hey @nitya-signoz thanks the severity text is working now after json parsing. Now just the IN not working on body attributes is the issue.

Can you create an issue on github, seems like json search for IN operator is broken

Hey @nitya-signoz any idea how I can use logs processors to also have another severity parser to parse from log.iostream variable? My plan is to update all severity_text where its an empty string based on log_iostream (stdout, stderr) values.. But whenever i try to use log.iostream I get this error:

{
  "status": "error",
  "errorType": "bad_data",
  "error": "could not simulate log pipelines processing.\nCollector errors: could not create logs processing simulator: failed to parse collector config: cannot unmarshal the configuration: decoding failed due to the following error(s):\n\nerror decoding 'processors': error reading configuration for \"signozlogspipeline/pipeline_EmptySeverityFix\": decoding failed due to the following error(s):\n\nerror decoding 'operators[1]': unmarshal to severity_parser: decoding failed due to the following error(s):\n\nerror decoding 'parse_from': unrecognized prefix"
}

Pipeline :

So

log.iostream

is something inside json body ? if yes you will need to have a json parser before it. You can specify the field inside a body directly . Please check this doc on how to parse json to a temp attribute and then use it https://signoz.io/docs/logs-pipelines/guides/json/

nope its in attributes

then it will be

attributes["log.iostream"]

Something like this

yep, try the above

ahh this works I was earlier trying attributes.log.iostream seems like dot notation doesn't work