This message was deleted.
# supports
Slackbot
11/28/2022, 9:09 AMThis message was deleted.
p
Palash Gupta
11/28/2022, 9:10 AM@nitya-signoz
š 1
n
nitya-signoz
11/28/2022, 9:13 AMFrom the operators that you have used, the timestamp is parsed by the regex, but you will have to use timestamp parser to populate the value eg:- https://github.com/SigNoz/nginx-logs-parsing/blob/c14a2178a050293a50549b79991ed3858e295c78/clickhouse-setup/otel-collector-config.yaml#L34
Right now your timestamp is getting set to epoch 0.
š 1
r
Rahul v
11/28/2022, 9:15 AMThanks for the quick response, sure I will chain more operators to get the final output but from the first transformation, shouldn't I see something in the UI?
I was expecting to see 3 attributes with string values named:
ā¢ log_timestamp
ā¢ log_entry_type
ā¢ log_msg
n
nitya-signoz
11/28/2022, 9:16 AMAre you able to see the attributes on the left side in interesting fields?
r
Rahul v
11/28/2022, 9:17 AMyes I do
n
nitya-signoz
11/28/2022, 9:18 AMIf your log is ingested you will be able to see logs when you change the time range in top right to something like Dec 1969 to current. As epoch 0 corresponds to 1 January 1970
r
Rahul v
11/28/2022, 9:22 AMok, I set the time frame to what you suggested, but still no log entries
what container logs should I look at to confirm log entries were consumed?
Rahul v
11/28/2022, 9:26 AMdo the otel-collector logs show this info?
the last log entry says:
Copy code
2022-11-28T09:03:39.192Z info fileconsumer/file.go:159 Started watching file {"kind": "receiver", "name": "filelog/cav", "pipeline": "logs", "component": "fileconsumer", "path": "/testlog/interface.mod.log"}n
nitya-signoz
11/28/2022, 9:26 AMYou can exec into the clickhouse pod and check
Copy code
docker exec -it clickhouse-setup-clickhouse-1 /bin/bash
clickhouse client
select * from signoz_logs.logs limit 5;nitya-signoz
11/28/2022, 9:27 AMYou should see the logs, would suggest you to use the time parser after that.
r
Rahul v
11/28/2022, 9:27 AMok, let me try
Rahul v
11/28/2022, 9:41 AMThanks @nitya-signoz
parsing the timestamp made the logs show up, they were ingested but not showing up in the UI.
This did the trick
Copy code
- type: regex_parser
regex: '^(?P<log_timestamp>\d{4}-\d{2}-\d{2}\s+\d{2}:\d{2}:\d{2}[,.]\d{3})\s+-\s+(?P<log_entry_type>\w+)\s+-\s+(?P<log_msg>.*)$'
timestamp:
parse_from: attributes.log_timestamp
layout_type: strptime
layout: '%Y-%m-%d %H:%M:%S,%f'š 1
6 Views