#support
Title
# support
n
Noor Ali
10/26/2023, 7:34 PMHello Support what can cause Alert in signoz not to trigger we are almost getting done with our POC and Demo just please answer my last question thanks for answering my questions before .
s
Srikanth Chekuri
10/27/2023, 11:41 AMYou can get rid of body EXISTS because each log will have body. What are you trying with group by
bodytimestamp_fieldn
Noor Ali
10/27/2023, 12:23 PMYes body and timestamp-field will work to get alert triggered
Hello Srikanth I am testing the following to get alert triggered
let me know which will work
s
Srikanth Chekuri
10/30/2023, 4:55 PMThe alert runs every 1 minute and only considers the last 5 minutes of data (with a max configurable time of 24 hours) by default. It will only trigger if the condition evaluates to true for evaluation period.
n
Noor Ali
10/30/2023, 4:57 PMHello Srikanth how do you write query for it
do make changes here for it elect
toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 30 MINUTE) AS interval,
toFloat64(count()) as value
FROM signoz_logs.distributed_logs
WHERE timestamp BETWEEN {{.start_timestamp_nano}} AND {{.end_timestamp_nano}}
GROUP BY interval;
-- available variables:
-- {{.start_timestamp_nano}}
-- {{.end_timestamp_nano}}
-- required columns (or alias):
-- value
-- interval
do we need to make any change in alerts.yml groups:
- name: ExampleCPULoadGroup
rules:
- alert: HighCpuLoad
expr: system_cpu_load_average_1m > 0.1
for: 0m
labels:
severity: warning
annotations:
summary: High CPU load
description: "CPU load is > 0.1\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
or do we need to make any changes in alertmanger.yml I just want to make sure
s
Srikanth Chekuri
10/30/2023, 5:04 PMYou don't need to change anything. Use the query builder to set up alert.
n
Noor Ali
10/30/2023, 5:05 PMI did but my alerts don't trigger at all
s
Srikanth Chekuri
10/30/2023, 5:07 PMShow me the how you created/updated it and show me some samples of recent logs when you expected the alert but it didn't trigger.
n
Noor Ali
10/30/2023, 5:08 PMhere are the logs 1695514384 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled.
1695514384 00000001 AdminTool A ADMU3200I: Server launched. Waiting for initialization status.
1695514492 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991150
1698342160 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991151
1698342160 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991152
1698354023 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991153
1698354023 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991154
log:
Java Home = /opt/IBM/WebSphere/Ap1644436278 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled.
timestamp:
2023-10-27T092211-05:00
log:
Java Home = /opt/IBM/WebSphere/Ap1644436278 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled.
timestamp:
2023-10-27T091701-05:00
log:
Java Home = /opt/1638963361 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 8192488
timestamp:
2023-10-27T091052-05:00
log:
Java Home = /opt/IBM/WebSphere/Ap1644436278 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled.
Screenshot 2023-10-30 at 12.10.00 PM.png
this is what is see now
s
Srikanth Chekuri
10/30/2023, 5:12 PMAre there any logs that match the condition in the last five minutes otherwise it is correct.
n
Noor Ali
10/30/2023, 5:12 PMwe are using this eceivers:
filelog:
include: [ /tmp/NewLogTest1.log]
start_at: beginning
operators:
- type: regex_parser
regex: '^(?P<timestamp_field>(\d+))(?P<body>.*)'
timestamp:
layout_type: epoch
layout: s
parse_from: attributes.timestamp_field
exporters:
clickhouselogsexporter:
dsn: tcp://clickhouse:9000/
docker_multi_node_cluster: ${DOCKER_MULTI_NODE_CLUSTER}
timeout: 5s
sending_queue:
queue_size: 100
retry_on_failure:
enabled: true
initial_interval: 5s
max_interval: 30s
max_elapsed_time: 300s
service:
pipelines:
logs:
receivers: [filelog]
exporters: [clickhouselogsexporter]
s
Srikanth Chekuri
10/30/2023, 5:13 PMAre there any logs that match the condition in the last five minutes?
n
Noor Ali
10/30/2023, 5:15 PMYes I just made change in the log
Here is my change in the log 1698685986 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991155
s
Srikanth Chekuri
10/30/2023, 5:16 PMWhat is the alert query?
n
Noor Ali
10/30/2023, 5:18 PMare you referring to this alert: LogMessageAlert
expr: count_over_time({logbody="ADMU3000I: Server igawas02 open for e-business"} == 1) by (timestamp_field) > 0
for: 1m
labels:
severity: critical
annotations:
summary: Log Message Detected
description: "The log message was detected: 'ADMU3000I: Server igawas02 open for e-business'"
s
Srikanth Chekuri
10/30/2023, 5:19 PMWhere is this count_over_time coming from? These are logs
Don't create a metric based alert for logs
n
Noor Ali
10/30/2023, 5:20 PMI am trying to do log based alert now as test
I get the above query when I press this button
If you can provide me the answer I can forward it my architect in their team my last on this project is Tuesday I am doing my final documentation thanks
s
Srikanth Chekuri
10/30/2023, 5:24 PM1. Use query builder
2. Upgrade signoz version so the default is query builder
n
Noor Ali
10/30/2023, 5:24 PMAlso my local host is over 30 days usage now is that cause of it i just want to make sure
s
Srikanth Chekuri
10/30/2023, 5:25 PMNo
n
Noor Ali
10/30/2023, 5:25 PMGot I will forward your answer to my Architect