Pavan Kumar
10/02/2024, 1:18 PMversion: '3.8'
services:
foo:
image: ...
...
bar:
image: ...
...
I want to collect logs without changing any micro-service service code. I am using logspout
to forward log with following command. This is successfully forwarding logs.
docker run --net=host --rm --name="logspout" \
--volume=/var/run/docker.sock:/var/run/docker.sock \
-e PORT=90 \
gliderlabs/logspout \
syslog+tcp://<host>:2255
What is missing is logs are not categorised in SigNoz under their respective service names. How to configure logspout
to send service
name along with logs. Or is there any other log collector other than logsspout
which can send more metadata along with logs.nitya-signoz
10/03/2024, 5:06 AMservice.name
or extract it from the body if possible. You can use SigNoz pipelines for that https://signoz.io/docs/logs-pipelines/introduction/
Even vector has support for similar attributes https://vector.dev/docs/reference/configuration/sources/docker_logs/#output-dataPavan Kumar
10/03/2024, 7:33 AMlogspout
which sends logs in OTel format directly to SigNoz with appropriate service name and env attribute?nitya-signoz
10/03/2024, 7:43 AMPavan Kumar
10/03/2024, 7:45 AMnitya-signoz
10/03/2024, 7:47 AMPavan Kumar
10/03/2024, 8:32 AMenv
explicitly. The service
is automatically picked up.
docker run -d --name datadog-agent \
--cgroupns host --pid host \
-e DD_LOGS_ENABLED=true \
-e DD_LOGS_CONFIG_CONTAINER_COLLECT_ALL=true \
-e DD_ENV=prod \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /var/lib/docker/containers:/var/lib/docker/containers:ro \
-v /proc/:/host/proc/:ro \
-v /opt/datadog-agent/run:/opt/datadog-agent/run:rw \
-v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro <http://gcr.io/datadoghq/agent:latest|gcr.io/datadoghq/agent:latest>
Interestingly, service name picked up by datadog is name specified in compose file, not
the container name.
For instance, in following example compose file we are running service foo
with replica
of 2. Each container name might be foo-foo-1
and foo-foo-2
. But datadog picks up service name as foo
which makes filtering log easy.
version: '3.8'
services:
foo:
image: ...
deploy:
replica: 2
Pavan Kumar
10/03/2024, 1:47 PMlogspout-signoz
or logspout-otel
nitya-signoz
10/03/2024, 2:48 PMPavan Kumar
10/03/2024, 4:55 PMPavan Kumar
10/04/2024, 6:53 AM[
{
"trace_id": "000000000000000018c51935df0b93b9",
"span_id": "18c51935df0b93b9",
"trace_flags": 0,
"severity_text": "error",
"severity_number": 5,
"attributes": {
"method": "GET",
"path": "/api/users",
"service": "bar",
"environment": "env.att"
},
"resources": {
"service.name": "name.with.dot",
"environment": "res1",
"namespace": "prod"
},
"message": "This is a log line error",
"environment":"TopE"
}
]
nitya-signoz
10/04/2024, 7:13 AMdeployment.environment
, you can use it instead of environment
Pavan Kumar
10/04/2024, 7:14 AMnitya-signoz
10/04/2024, 7:15 AMPavan Kumar
10/04/2024, 7:20 AMnitya-signoz
10/04/2024, 7:27 AMdeployment.environment
and not environment
. Also please make sure that ``deployment.environment` is added to resource attribute and not attribute. Here you have added env to both https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1728024791870019?thread_ts=1727875092.767829&cid=C01HWQ1R0BCPavan Kumar
10/13/2024, 6:17 AMlogspout-signoz
extension which can collect logs and send it to signoz endpoints. Following are the new features added on top of logspout
.
1. Direct post to signoz http
endpoint. So this extension can send more detailed logs.
2. Auto detect service name, so no special configuration needed.
a. For JSON logs, picks name from JSON service
field.
b. Otherwise pick service name from docker-compose
service name.
c. Otherwise use docker image name
as service name
3. Auto detect env name, so no special configuration needed
a. For JSON logs, picks name from JSON env
field.
b. Otherwise pick env
from logspout-signoz
env variable ENV
.
4. Auto parse JSON logs.
a. Map well known JSON log attribute to appropriate Signoz log payload fields. e.g level
to SeverityText
, etc
b. Pack other JSON attribute to into attributes
key of Signoz log payload.
I hope this logspout-signoz
extension will be helpful for others as well, given it provides zero configuration log collection from docker/docker-compose environment. So could you please also add following command to your documentation present at this link so that people could start using it?
docker run -d \
--volume=/var/run/docker.sock:/var/run/docker.sock \
-e 'SIGNOZ_LOG_ENDPOINT=<http://192.168.0.104:8082>' \
-e 'ENV=prod' \
pavanputhra/logspout-signoz \
<signoz://localhost:8082>
Link to GitHub code: https://github.com/pavanputhra/logspout-signoz.nitya-signoz
10/13/2024, 2:38 PMPavan Kumar
10/22/2024, 10:21 AMnitya-signoz
10/23/2024, 4:56 PMnitya-signoz
11/01/2024, 5:23 AMnitya-signoz
11/01/2024, 5:25 AMPavan Kumar
11/01/2024, 5:32 AMnitya-signoz
11/01/2024, 5:33 AMPavan Kumar
11/01/2024, 5:40 AMYuvraj Singh
11/01/2024, 6:52 AMPavan Kumar
11/01/2024, 8:36 AMYuvraj Singh
11/01/2024, 9:01 AMPavan Kumar
11/06/2024, 5:17 PMYuvraj Singh
11/11/2024, 9:09 AMPavan Kumar
11/26/2024, 1:00 PMYuvraj Singh
11/26/2024, 1:00 PMPavan Kumar
11/26/2024, 1:08 PMnitya-signoz
11/26/2024, 1:54 PM