https://signoz.io
Join Slack
This message was deleted.
# support
s
This message was deleted.
p
@nitya-signoz
šŸ‘ 1
n
From the operators that you have used, the timestamp is parsed by the regex, but you will have to use timestamp parser to populate the value eg:- https://github.com/SigNoz/nginx-logs-parsing/blob/c14a2178a050293a50549b79991ed3858e295c78/clickhouse-setup/otel-collector-config.yaml#L34 Right now your timestamp is getting set to epoch 0.
šŸ™Œ 1
r
Thanks for the quick response, sure I will chain more operators to get the final output but from the first transformation, shouldn't I see something in the UI? I was expecting to see 3 attributes with string values named: ā€¢ log_timestamp ā€¢ log_entry_type ā€¢ log_msg
n
Are you able to see the attributes on the left side in interesting fields?
r
yes I do
n
If your log is ingested you will be able to see logs when you change the time range in top right to something like Dec 1969 to current. As epoch 0 corresponds to 1 January 1970
r
ok, I set the time frame to what you suggested, but still no log entries what container logs should I look at to confirm log entries were consumed?
do the otel-collector logs show this info? the last log entry says:
Copy code
2022-11-28T09:03:39.192Z        info    fileconsumer/file.go:159        Started watching file   {"kind": "receiver", "name": "filelog/cav", "pipeline": "logs", "component": "fileconsumer", "path": "/testlog/interface.mod.log"}
n
You can exec into the clickhouse pod and check
Copy code
docker exec -it clickhouse-setup-clickhouse-1 /bin/bash

clickhouse client

select * from signoz_logs.logs limit 5;
You should see the logs, would suggest you to use the time parser after that.
r
ok, let me try
Thanks @nitya-signoz parsing the timestamp made the logs show up, they were ingested but not showing up in the UI. This did the trick
Copy code
- type: regex_parser
      regex: '^(?P<log_timestamp>\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}[,.]\d{3})\s+-\s+(?P<log_entry_type>\w+)\s+-\s+(?P<log_msg>.*)$'
      timestamp:
        parse_from: attributes.log_timestamp
        layout_type: strptime
        layout: '%Y-%m-%d %H:%M:%S,%f'
looks like the first operation requires the timestamp, will remember that, cheers
šŸ‘ 1
5 Views
Open in Slack
PreviousNext
Powered by

SigNoz Community

SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.

Powered by