https://signoz.io logo
#support
Title
# support
k

Krishnadas K P

07/26/2023, 8:16 AM
I am trying to get logs from logstash to Signoz. Logstash is installed directly in the ubuntu VM and signoz is installed using docker in the same VM. But I am unable to get any logs in Signoz via open telemetry collector. This is my logstash config
input {
file {
path => "/home/iislogs.log"
start_position => "beginning"
}
}
output {
tcp {
port => 2255
host => "localhost"
}
}
Signoz setup is done as per this doc https://signoz.io/docs/userguide/logstash_to_signoz/ What must I be doing wrong here ? Please help
p

Prashant Shahi

07/26/2023, 8:40 AM
Did you restart otel-collector properly after config change and exposed ports?
k

Krishnadas K P

07/26/2023, 8:42 AM
Yes. I had restarted logstash and also docker compose down and up all containers
Are we supposed to see any logs in otel collector when it is receiving data from a receiver ? Because I am not seeing any logs for that yet
@Prashant Shahi I have set logstash to send data to localhost:2255 and set signoz to receive data at 0.0.0.0:2255.. Is this correct ? I can see in logstash logs that it is opening connection correctly. But no movement in signoz otel collector logs
@Prashant Shahi I have also tried to test logs from mounted volume as per https://signoz.io/docs/userguide/collect_logs_from_file/ This is not showing up in Signoz UI either. I can see the following log in otel collector container. But nothing more. 2023-07-26T094823.883Z info fileconsumer/file.go:192 Started watching file {"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "/tmp/python.log"}
Curiously enough the docker container logs are working fine and getting exported to clickhouse logs and visualized in signoz UI. Both logstash and filelog receivers that I added arent working
@Prashant Shahi, @nitya-signoz Any idea on this please ?
n

nitya-signoz

07/26/2023, 3:29 PM
• For file log receiver, are new logs getting written to the file, if not then you will have to use the
start_at
: beginning configuration.https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/filelogreceiver • For logstash can you share what are the logs of logstash ? also have you exposed the port of otel-collector ?
k

Krishnadas K P

07/26/2023, 3:50 PM
Thanks for the reply @nitya-signoz. 1. Yes. New logs are getting written. I am using the same tutorial of python logging as given in Signoz docs 2. Yes. I have exposed the 2255 port on otel collector. Logstash log snippet is attached.
n

nitya-signoz

07/26/2023, 3:52 PM
• Can you check if you have mounted your file properly ? • Also please share your otel-collector config
k

Krishnadas K P

07/26/2023, 3:57 PM
1. Yes I can see the file after exec ing into the docker container. 2. Attached.
n

nitya-signoz

07/26/2023, 3:59 PM
This seems correct, what environment are you running this ? linux ?
k

Krishnadas K P

07/26/2023, 4:00 PM
Yes. In Ubuntu 20.04.6 LTS (Focal Fossa)
Any idea @nitya-signoz ? I would have thought the filelog receiver would have been fairly simple to work with :(
p

Prashant Shahi

07/27/2023, 1:28 AM
@Krishnadas K P updating content of already mounted otelcol-config file is not often updated in the restarted container. hard remove of the otel-collector container is required for that.
Copy code
docker stop <otelcollector-container>
docker rm <otelcollector-container>


docker-compose -f ...

OR

./install.sh
k

Krishnadas K P

07/27/2023, 1:40 AM
I was always doing docker-compose down and then docker-compose up -d. Funnily enough, I removed the existing docker container logs receiver from the otel-collector-config and just added the filelog receiver for my logs folder and it started working. I will play around a bit and confirm what changed
2 Views