Hi @User - I created a webhook and defined a basic rule. But the alert is not triggered anytime - not sure what I am missing here. Attached screenshot of rule definition and logs from alert-manager. I dont find any triggered alerts, so far. I kept value to greater than -1 - so that i can see some alert atleast. But till now, no luck. Note - the alert is in firing status. Please do the needful. Thanks!

Hi @User! We encountered this issue yesterday and currently investigating it. This issue is happening on k8s. Using the Docker, alerts works seemlessly. We will be creating a fix for this soon.

Thanks @User - will wait for k8s fix.

@User SigNoz v0.7.5 is released and along with that chart signoz-0.0.12 is also out with the alerts fix. Refer here to upgrade: https://signoz.io/docs/operate/kubernetes/#upgrade

thanks @User - installed latest chart. Now I am getting a new error while accessing metrics, traces or any page. But Alerts page is fine - able to add/delete my alerts. IP mentioned in the error (refer screenshot) doesnt seem to be my cluster IP. Please help me in troubleshooting this. Note: Agents are ingesting - and no error seen at agent side. All pods after chart install are in running status. Thanks!

It looks like clickhouse pod is not up anymore. Can you check if clickhouse statefulset is running?

Hi @User - found the problem. Yes, Chi cluster was not initiated earlier - but not sure about real reason. Now, I enabled nameOverride and fullnameOverride fields in chart (signoz service)- and its working fine. Hope this is not fix - but not sure too 🙂 Able to connect fine now. Thanks!