Hi Team, how can we remove test data? my actual lo...
# support
s
Hi Team, how can we remove test data? my actual logs is getting mixed with test logs, it's difficult to find actual logs, please help
a
@nitya-signoz
n
By test data do you mean the default logs that are getting collected ? Also are you running SigNoz on docker or kubernetes ?
s
yes default logs, they're coming up even when i go live
I am running on docker
n
Just remove
filelog/dockercontainers
from the config to stop default docker logs collection https://github.com/SigNoz/signoz/blob/a3c917cca0b1c54e0f52cc1fc1440bc721c4e611/deploy/docker/clickhouse-setup/otel-collector-config.yaml#L208 and restart otel-collector
s
Okay, let me try
n
How are you sending your application logs and what is your tech stack ?
s
sending from a nodejs application
just adding it to console and signoz is picking it up
i am able to see my logs just that lot of test data is also seen
n
Got it the above change not collector your applications logs as well, as by default docker doesn’t support names in log file path like kubernetes. There are a few ways to solve this problem
s
yeah I restarted the container, not able to see my logs as well
should i revert that change?
n
• Use logs sdk to send logs directly from your application to signoz https://github.com/open-telemetry/opentelemetry-js • Use this solution https://github.com/SigNoz/signoz/issues/1597#issuecomment-1259234665 This will add container name to your logs though which you can filter your application logs in the UI.
What deployment method do your use to deploy your apps in prod? docker or k8s ? based on that I can suggest one of the above solutions
s
currently I am just working on a poc in my local, it'll mostly be using docker
we're using winston logger throughout so I don't think 1st solution is enough right?
n
Both of the solutions will work, but code level changes will be required for 1st and here is the example for that https://github.com/fuaiyi/opentelemetry-js/tree/0c1c95a1dfa15b4d0fcee52a9a572cd88dba876e/experimental/examples/logs . • If you will use docker in prod then go with the second solution. It will help you collect and filter logs of other applications as well.
s
yeah even I felt 2nd one is better
where can i find more details about it
n
The comment I have linked is self sufficient, let me know if you don’t understand something.
s
Okay sure, I'll have a look and get back to you
Thanks a lot!
@nitya-signoz the changes that you have showed should be made in which file?
n
The ote-collector-config.yaml file
s
okay got it
and the listen_address should be what?
can it be the same whatever you have mentioned
n
yes
s
@nitya-signoz I am getting the following error 2023/06/06 062351 !! dial tcp 0.0.0.02255 connect: connection refused
I think i need to export 2255
how can we do that in this yaml?
n
s
okay
the following is the output that i see now
<syslog+tcp://0.0.0.0:2255>
2023/06/06 103159 # logspout v3.2.14 by gliderlabs 2023/06/06 103159 # adapters: raw syslog tcp tls udp multiline 2023/06/06 103159 # options : 2023/06/06 103159 persist:/mnt/routes 2023/06/06 103159 # jobs : http[routes,health,logs]:80 pump routes 2023/06/06 103159 # routes : # ADAPTER ADDRESS CONTAINERS SOURCES OPTIONS # syslog+tcp 0.0.0.0:2255 map[] 2023/06/06 103159 http[logs,routes,health]:80 ended: listen tcp 0.0.0.080 bind: address already in use
n
Give me a few mins, I am checking this.
s
thank you!
n
I am able to run it fine, lets do a 5 mins huddle
s
yes
n
Copy code
sudo docker run --rm --name="logspout"         --volume=/var/run/docker.sock:/var/run/docker.sock         gliderlabs/logspout         <syslog+tcp://192.168.0.101:2255>