This message was deleted.
# support
s
This message was deleted.
n
The config is written for general use case to ingest k8s pod logs. The config doesn’t parse the log itself. If you want to parse all your json logs you should apply more operators ex: in this case I wanted to parse specific format of nginx logs https://github.com/SigNoz/nginx-logs-parsing/blob/c14a2178a050293a50549b79991ed3858e295c78/clickhouse-setup/otel-collector-config.yaml#L24 If you want to parse your json logs you can use a router to capture your json logs and then parse it using a pipeline https://github.com/SigNoz/logs-benchmark/blob/0b2451e6108d8fa5fdd5808c4e174bd52b9d55d3/signoz/signoz-client/otel-collector-config.yaml#L12
Even the routing choice here is super-vague by assuming (incorrectly) that any line beginning with a open-curly-brace is in some parse-able JSON format: https://github.com/SigNoz/charts/blob/main/charts/k8s-infra/values.yaml#L96-L104
This is incorrect. When your logs are stored by k8s into a file it follows a specific format for storing logs. There won’t be a case where your k8s pod logs are captured but it is not stored in docker or some other format.
n
Ok, so I can use
json_parser
to parse the
message.body
then I will have to find a way of applying a conditional expression that allows for different mapping rules dependent on the source app, for example where
message.body.xid
for one application would map to
session_id
and
message.body.xid
for another application would map to
transaction_id
.
n
Right, you can change and create mappings according to your business logic using those pipelines. We also have a UI-based parser coming up but is delayed due to priorities. https://github.com/SigNoz/signoz/pull/2457 But till then you can create them manually in your collector config. You can read more about them here https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/docs/types/pipeline.md. Try writing one using the example I shared and using the link above and let me know if you face any issues.