I am trying to get logs from logstash to Signoz. L...
# support
I am trying to get logs from logstash to Signoz. Logstash is installed directly in the ubuntu VM and signoz is installed using docker in the same VM. But I am unable to get any logs in Signoz via open telemetry collector. This is my logstash config
input {
file {
path => "/home/iislogs.log"
start_position => "beginning"
output {
tcp {
port => 2255
host => "localhost"
Signoz setup is done as per this doc https://signoz.io/docs/userguide/logstash_to_signoz/ What must I be doing wrong here ? Please help
Did you restart otel-collector properly after config change and exposed ports?
Yes. I had restarted logstash and also docker compose down and up all containers
Are we supposed to see any logs in otel collector when it is receiving data from a receiver ? Because I am not seeing any logs for that yet
@Prashant Shahi I have set logstash to send data to localhost:2255 and set signoz to receive data at Is this correct ? I can see in logstash logs that it is opening connection correctly. But no movement in signoz otel collector logs
@Prashant Shahi I have also tried to test logs from mounted volume as per https://signoz.io/docs/userguide/collect_logs_from_file/ This is not showing up in Signoz UI either. I can see the following log in otel collector container. But nothing more. 2023-07-26T094823.883Z info fileconsumer/file.go:192 Started watching file {"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "/tmp/python.log"}
Curiously enough the docker container logs are working fine and getting exported to clickhouse logs and visualized in signoz UI. Both logstash and filelog receivers that I added arent working
@Prashant Shahi, @nitya-signoz Any idea on this please ?
• For file log receiver, are new logs getting written to the file, if not then you will have to use the
: beginning configuration.https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver • For logstash can you share what are the logs of logstash ? also have you exposed the port of otel-collector ?
Thanks for the reply @nitya-signoz. 1. Yes. New logs are getting written. I am using the same tutorial of python logging as given in Signoz docs 2. Yes. I have exposed the 2255 port on otel collector. Logstash log snippet is attached.
• Can you check if you have mounted your file properly ? • Also please share your otel-collector config
1. Yes I can see the file after exec ing into the docker container. 2. Attached.
This seems correct, what environment are you running this ? linux ?
Yes. In Ubuntu 20.04.6 LTS (Focal Fossa)
Any idea @nitya-signoz ? I would have thought the filelog receiver would have been fairly simple to work with :(
@Krishnadas K P updating content of already mounted otelcol-config file is not often updated in the restarted container. hard remove of the otel-collector container is required for that.
Copy code
docker stop <otelcollector-container>
docker rm <otelcollector-container>

docker-compose -f ...


I was always doing docker-compose down and then docker-compose up -d. Funnily enough, I removed the existing docker container logs receiver from the otel-collector-config and just added the filelog receiver for my logs folder and it started working. I will play around a bit and confirm what changed