Hey all! Happy SigNoz Cloud user here with our team! We're experimenting with Alerts right now. We have all syslog messages in the Logs and would like to create alerts if a systemd service is failed. Basically we query

body CONTAINS .service: Failed with result

and group this by hostname. It works somehow for getting a count of messages, but it would be great to actually use the name of the failed service in the Alert/Notifications. Do we need to add some regex magic to the body of the alert or is there a even more elegant way to achieve such an alert?

serviceA, serviceB, serviceC have failed on hostA

You need to add the service to the group by clause of the query. This will get you the counts by service and evaluates the condition you set.

Seems like there is only

appname

and it's always

systemd

. That does not help that much.

Do you have any other attribute in your data that indicates the seviceA/B/C you mentioned above?

No, it's

systemd

that orchestrates those and marks them

failed

. Basically I have the

message

object that was parsed and has the service-name in it, so

[a-zA-Z0-9-_]+(?=\.)

would match the service name perfect for the messages:

"serviceA.service: Failed with result 'exit-code'."

How do I need to specify the

servicename

in the Alert to be picked up?

I didn't understand the question.

In the settings of the alert channel there is the possibility to adjust the template to your needs. Now: how can I access/select the new introduced field in the template?

I think I responded to same question on the intercom. You can use

{{.Labels.servicename}}