https://signoz.io logo
#support
Title
# support
n

Noor Ali

10/26/2023, 7:34 PM
Hello Support what can cause Alert in signoz not to trigger we are almost getting done with our POC and Demo just please answer my last question thanks for answering my questions before .
s

Srikanth Chekuri

10/27/2023, 11:41 AM
You can get rid of body EXISTS because each log will have body. What are you trying with group by
body
,
timestamp_field
?
n

Noor Ali

10/27/2023, 12:23 PM
Yes body and timestamp-field will work to get alert triggered
Hello Srikanth I am testing the following to get alert triggered
let me know which will work
s

Srikanth Chekuri

10/30/2023, 4:55 PM
The alert runs every 1 minute and only considers the last 5 minutes of data (with a max configurable time of 24 hours) by default. It will only trigger if the condition evaluates to true for evaluation period.
n

Noor Ali

10/30/2023, 4:57 PM
Hello Srikanth how do you write query for it
do make changes here for it elect toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 30 MINUTE) AS interval, toFloat64(count()) as value FROM signoz_logs.distributed_logs WHERE timestamp BETWEEN {{.start_timestamp_nano}} AND {{.end_timestamp_nano}} GROUP BY interval; -- available variables: -- {{.start_timestamp_nano}} -- {{.end_timestamp_nano}} -- required columns (or alias): -- value -- interval
do we need to make any change in alerts.yml groups: - name: ExampleCPULoadGroup rules: - alert: HighCpuLoad expr: system_cpu_load_average_1m > 0.1 for: 0m labels: severity: warning annotations: summary: High CPU load description: "CPU load is > 0.1\n VALUE = {{ $value }}\n LABELS = {{ $labels }}"
or do we need to make any changes in alertmanger.yml I just want to make sure
s

Srikanth Chekuri

10/30/2023, 5:04 PM
You don't need to change anything. Use the query builder to set up alert.
n

Noor Ali

10/30/2023, 5:05 PM
I did but my alerts don't trigger at all
s

Srikanth Chekuri

10/30/2023, 5:07 PM
Show me the how you created/updated it and show me some samples of recent logs when you expected the alert but it didn't trigger.
n

Noor Ali

10/30/2023, 5:08 PM
here are the logs 1695514384 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled. 1695514384 00000001 AdminTool A ADMU3200I: Server launched. Waiting for initialization status. 1695514492 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991150 1698342160 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991151 1698342160 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991152 1698354023 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991153 1698354023 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991154
log: Java Home = /opt/IBM/WebSphere/Ap1644436278 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled. timestamp: 2023-10-27T092211-05:00 log: Java Home = /opt/IBM/WebSphere/Ap1644436278 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled. timestamp: 2023-10-27T091701-05:00 log: Java Home = /opt/1638963361 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 8192488 timestamp: 2023-10-27T091052-05:00 log: Java Home = /opt/IBM/WebSphere/Ap1644436278 00000001 WorkSpaceMana A WKSP0500I: Workspace configuration consistency check is disabled.
Screenshot 2023-10-30 at 12.10.00 PM.png
this is what is see now
s

Srikanth Chekuri

10/30/2023, 5:12 PM
Are there any logs that match the condition in the last five minutes otherwise it is correct.
n

Noor Ali

10/30/2023, 5:12 PM
we are using this eceivers: filelog: include: [ /tmp/NewLogTest1.log] start_at: beginning operators: - type: regex_parser regex: '^(?P<timestamp_field>(\d+))(?P<body>.*)' timestamp: layout_type: epoch layout: s parse_from: attributes.timestamp_field exporters: clickhouselogsexporter: dsn: tcp://clickhouse:9000/ docker_multi_node_cluster: ${DOCKER_MULTI_NODE_CLUSTER} timeout: 5s sending_queue: queue_size: 100 retry_on_failure: enabled: true initial_interval: 5s max_interval: 30s max_elapsed_time: 300s service: pipelines: logs: receivers: [filelog] exporters: [clickhouselogsexporter]
s

Srikanth Chekuri

10/30/2023, 5:13 PM
Are there any logs that match the condition in the last five minutes?
n

Noor Ali

10/30/2023, 5:15 PM
Yes I just made change in the log
Here is my change in the log 1698685986 00000001 AdminTool A ADMU3000I: Server igawas02 open for e-business; process id is 15991155
s

Srikanth Chekuri

10/30/2023, 5:16 PM
What is the alert query?
n

Noor Ali

10/30/2023, 5:18 PM
are you referring to this alert: LogMessageAlert expr: count_over_time({logbody="ADMU3000I: Server igawas02 open for e-business"} == 1) by (timestamp_field) > 0 for: 1m labels: severity: critical annotations: summary: Log Message Detected description: "The log message was detected: 'ADMU3000I: Server igawas02 open for e-business'"
s

Srikanth Chekuri

10/30/2023, 5:19 PM
Where is this count_over_time coming from? These are logs
Don't create a metric based alert for logs
n

Noor Ali

10/30/2023, 5:20 PM
I am trying to do log based alert now as test
I get the above query when I press this button
If you can provide me the answer I can forward it my architect in their team my last on this project is Tuesday I am doing my final documentation thanks
s

Srikanth Chekuri

10/30/2023, 5:24 PM
1. Use query builder 2. Upgrade signoz version so the default is query builder
n

Noor Ali

10/30/2023, 5:24 PM
Also my local host is over 30 days usage now is that cause of it i just want to make sure
s

Srikanth Chekuri

10/30/2023, 5:25 PM
No
n

Noor Ali

10/30/2023, 5:25 PM
Got I will forward your answer to my Architect