A question: I setup an alert, which is based on traces. it's set to check in last 5 minutes, if anyt...
m
A question: I setup an alert, which is based on traces. it's set to check in last 5 minutes, if anything found in my query. Now Im asking how often signoz alert manager queries database for my alert? is it like every minute? can it be setup somewhere? Because I don't want to be alerted more than once, in the period of 5 minutes. Is that even possible?
s
Now Im asking how often signoz alert manager queries database for my alert? is it like every minute?
Yes, it is every minute
can it be setup somewhere?
Yes, it can be configured in the "More options" of the alert condition https://signoz.io/docs/alerts-management/trace-based-alerts/#step-2-define-alert-conditions
🙌 1
m
Is there a way to not send "Resolved" message? IMO It's not a correct assumption if a trace's threshold is not satisfied anymore, it's resolved.
s
It's a correct assumption. However, for whatever reason, if you don't want to receive a resolved notification, you can toggle the settings in the alert channel setting.
✅ 1
m
It's a correct assumption
How so? maybe Im using it in weird way. But this is basically one of my queries:
hasError = true | messaging.kafka.message.topic EXISTS
Threshold is > 0, in internal of 5 minutes. So basically, once any unhandled exception in our kafka consuming services occurs, this trigger "Resolving it" I guess would mean when I'm making a release, and this exception would be gone after a defined interval. But that's PER exception type.
I found "exception based" less useful, because I need the combiniaiton of "event" and "trace", to query ONLY for events that are unhandled when processing a kafka message. That's possible to query on Clickhouse, but not using Signoz UI.