https://signoz.io logo
Join Slack
Powered by
Hi. :slightly_smiling_face: We build self hosted S...
# support
k
Hi. 🙂 We build self hosted Signoz using Linux binary. And currently we are trying configure some alerts send by email. Below is our systemd.env configuration:
Copy code
SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=info
INVITE_EMAIL_TEMPLATE=/opt/signoz/templates/email/invitation_email_template.html
SIGNOZ_SQLSTORE_SQLITE_PATH=/var/lib/signoz/signoz.db
SIGNOZ_WEB_ENABLED=true
SIGNOZ_WEB_DIRECTORY=/opt/signoz/web
SIGNOZ_JWT_SECRET=secret
SIGNOZ_ALERTMANAGER_PROVIDER=signoz
SIGNOZ_TELEMETRYSTORE_PROVIDER=clickhouse
SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=<tcp://localhost:9000>
DOT_METRICS_ENABLED=true
SIGNOZ_EMAILING_ENABLED=true
SIGNOZ_EMAILING_SMTP_ADDRESS=smtp.domain:25
SIGNOZ_EMAILING_SMTP_TLS_ENABLED=false
SIGNOZ_EMAILING_SMTP_FROM=test@domain.com
SIGNOZ_EMAILING_TEMPLATES_DIRECTORY=/opt/signoz/templates
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__FROM=test@domain.com
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__SMARTHOST=smtp.domain:25
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__REQUIRE__TLS=false
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__TLS__CONFIG__INSECURE__SKIP__VERIFY=true
Sending the invitation works fine for this SMTP serwer. 🙂 But for alerts we have some problems. After restarting the Signoz, in logs we see that it using some localhost server:
Copy code
:{alertname=\"Test alertaaaaa\"}","attempts":1,"err":"establish connection to server: dial tcp 127.0.0.1:25: connect: connection refused"}
From UI perspective we see:
Copy code
'require_tls' is true (default) but "smtp.stepstone.tools:25" does not advertise the STARTTLS extension
could you please help us? What could be wrong?
v
Try re-saving the notification channel now.
From the UI, go the email alert channel and click Save again
k
It's the same 😞
l
Any ideas here?
v
Copy code
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__REQUIRE__TLS=false
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__TLS__CONFIG_INSECURE__SKIP__VERIFY=true
Try this please, there seems to be 2 underscores in what you sent.
k
I also tried it.
And result is the same 😞
v
Okay...
Can you send me the dump of 
SELECT * FROM alertmanager_config
?
(It might contain sensitive info, so feel free to redact that)
l
That query should be run inclickhouse?
v
No inside sqlite. If you are using docker, you can exec into the signoz container and run 
apk add sqlite
and then 
sqlite3 /var/lib/signoz/signoz.db
. Then run the SELECT query
If you are using k8s, just exec into the pod
l
We set it up as binary on ec2.
v
Perfect, easier then. Do you see this existing 
/var/lib/signoz/signoz.db
?
l
yes
v
Copy code
sudo apt-get install sqlite3
sqlite3 /var/lib/signoz/signoz.db
l
I masked our data. I'm surprised that there is require tls to true. As mentioned above we set environment variables to not use tls.
alertmanager.yaml
v
Copy code
"smtp_from": "<mailto:mymail@DOMAIN.com|mymail@DOMAIN.com>",
      "smtp_hello": "localhost",
      "smtp_smarthost": "smtp.domain.tools:25",
      "smtp_require_tls": true,
      "smtp_tls_config": { "insecure_skip_verify": true },
out of these others are set to your env variable apart from smtp_require_tls?
l
here are env variables
Copy code
SIGNOZ_INSTRUMENTATION_LOGS_LEVEL=info
INVITE_EMAIL_TEMPLATE=/opt/signoz/templates/email/invitation_email_template.html
SIGNOZ_SQLSTORE_SQLITE_PATH=/var/lib/signoz/signoz.db
SIGNOZ_WEB_ENABLED=true
SIGNOZ_WEB_DIRECTORY=/opt/signoz/web
SIGNOZ_JWT_SECRET=secret
SIGNOZ_ALERTMANAGER_PROVIDER=signoz
SIGNOZ_TELEMETRYSTORE_PROVIDER=clickhouse
SIGNOZ_TELEMETRYSTORE_CLICKHOUSE_DSN=<tcp://localhost:9000>
DOT_METRICS_ENABLED=true
SIGNOZ_EMAILING_ENABLED=true
SIGNOZ_EMAILING_SMTP_ADDRESS=smtp.domain:25
SIGNOZ_EMAILING_SMTP_TLS_ENABLED=false
SIGNOZ_EMAILING_SMTP_FROM=test@domain.com
SIGNOZ_EMAILING_TEMPLATES_DIRECTORY=/opt/signoz/templates
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__FROM=test@domain.com
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__SMARTHOST=smtp.domain:25
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__REQUIRE__TLS=false
SIGNOZ_ALERTMANAGER_SIGNOZ_GLOBAL_SMTP__TLS__CONFIG__INSECURE__SKIP__VERIFY=true
v
And your version is?
l
0.89.0
v
Found the issue,
let me raise a fix for you guys
l
Many thanks!
Could you share link to posted issue? Just to track progress
v
I dont know how it is working for these folks: https://github.com/SigNoz/signoz/issues/7009
13 Views
Open in Slack
PreviousNext
Powered by