https://signoz.io logo
Join Slack
Powered by
Hello everyone. I'm sending system logs to SigNoz ...
# support
e
Hello everyone. I'm sending system logs to SigNoz through the 
syslog
module in Python. I followed the documentation and then sent the logs through a Pyhon script. When I go into 
localhost:3301/logs
and search for the keyword I used in my log message, I see a couple of results but with a body that is different than my original message. Any idea what goes wrong here?
a
@nitya-signoz can you look into this?
n
Can you provide more info on what you were expecting and what you got? Also, do check out the protocol for Syslog, a mismatch on either side might cause a problem https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/syslogreceiver#configuration
e
Sure. For example, this is the script I used:
Copy code
import logging
from logging.handlers import SysLogHandler


def main():
    logger = logging.getLogger(__name__)
    logger.setLevel(logging.DEBUG)
    handler = SysLogHandler(
        facility=SysLogHandler.LOG_DAEMON,
        address='/dev/log'
        )

    formatter = logging.Formatter(
        fmt="%(asctime)s - %(filename)s:%(funcName)s:%(lineno)d %(levelname)s - '%(message)s'",
        datefmt="%Y-%m-%d %H:%M:%S"
        )
    handler.setFormatter(formatter)
    logger.addHandler(handler)

    logger.debug('Sending syslog_message to SigNoz!')

if __name__ == '__main__':
    main()
As you can see, this is the message 
Sending syslog_message to SigNoz!
I want to send to SigNoz. After I run this script, I search for the keyword 
syslog_message
and I get nothing in return. I've looked into the documentation of SysLogHandler and it looks like the protocol is RFC 5424: https://docs.python.org/3/library/logging.handlers.html#sysloghandler (not sure though) If the protocol is correct for the logging module. How can I configure the one on SigNoz?
n
Will get back to you on this by today EOD.
So if you check the syslog file it’s getting written as
Copy code
Dec  2 10:43:50 ip-172-31-27-188 systemd[631729]: Listening on D-Bus User Message Bus Socket.
Dec  2 10:43:50 ip-172-31-27-188 systemd[631729]: Reached target Sockets.
Dec  2 10:43:50 ip-172-31-27-188 systemd[631729]: Reached target Basic System.
Dec  2 10:43:50 ip-172-31-27-188 systemd[631729]: Reached target Main User Target.
Dec  2 10:43:50 ip-172-31-27-188 systemd[631729]: Startup finished in 45ms.
Dec  2 10:43:50 ip-172-31-27-188 systemd[1]: Started User Manager for UID 1000.
Dec  2 10:43:50 ip-172-31-27-188 systemd[1]: Started Session 81 of User ubuntu.
Dec  2 10:49:34 ip-172-31-27-188 2022-12-02 10:49:34 - main.py:main:20 DEBUG - 'Sending syslog_message to SigNoz!'
Dec  2 10:49:46 ip-172-31-27-188 systemd[1]: Started Session 83 of User ubuntu.
Dec  2 10:49:53 ip-172-31-27-188 2022-12-02 10:49:53 - main.py:main:20 DEBUG - 'Sending syslog_message to SigNoz!'
Now the Syslog parser is putting
Copy code
`2022-12-02 10:49:53 - main.py:main:20 DEBUG - 'Sending syslog_message to SigNoz!'
in the value of the body. Which I think is your concern right? If yes the reason here is in the syslog format eveything that you have published from the python script goes to the body as it writes its own timestamp. Incase you want to parse the above and only keep
Copy code
`'Sending syslog_message to SigNoz!'
as your body you will have pipeline operators to parse your logs in the otel-collector-config. ex:- https://github.com/SigNoz/nginx-logs-parsing/blob/c14a2178a050293a50549b79991ed3858e295c78/clickhouse-setup/otel-collector-config.yaml#L30
e
@nitya-signoz I'm sorry for replying too late. So why do you think searching for that specific keyword is not shown in the SigNoz dashboard? Is it because the parser didn't get that specific format?
n
While searching it should work regardless if the keyword is present in the log body. Can you check if the correct timerange is selected and try once again and check ?
e
@nitya-signoz, I'll try to list what I've done step by step, and let me know please which step is not clear
n
We can get on a call for this, let me know which timezone you are in ?
e
Thanks, I've scheduled it.
103 Views
Open in Slack
PreviousNext
Powered by