https://signoz.io logo
#support
Title
# support
w

Wejdan

10/30/2022, 9:39 AM
i tried to connect the logs in kubernetes but i faced this error no files match the configured include patterns {"kind": "receiver", "name": "filelog/k8s", "pipeline": "logs", "component": "fileconsumer", "include": ["/var/log/pods/*/*/*.log"], "exclude": []} could anyone help me?
n

nitya-signoz

10/31/2022, 1:56 AM
Can you give us more info about your kubernetes deployment like you are using EKS or something like minikube ?
w

Wejdan

10/31/2022, 7:53 AM
no, its oracle cloud and it's connected with open telemetry
the tracer is good but logs show errors
n

nitya-signoz

10/31/2022, 2:38 PM
Hi, we can you please exec into the OTEL collector pod and check log files are present inside
/var/log/pods/<pod_name>/<container_name>/*.log
also check the permissions of the log file. We use the root user to read these files so might be some issue with that.
w

Wejdan

11/01/2022, 10:22 AM
filelog/k8s:
include:
# Include logs from all container
- /var/log/pods/<pod_name>/<container_name>/*.log
exclude:
# Exclude logs from all containers from kube-system namespace
- /var/log/pods/kube-system_*/*/*.log
# Exclude logs from all hotrod containers
- /var/log/pods/*_hotrod-*/*/*.log
- /var/log/pods/*_locust-*_*/*/*.log
start_at: beginning
include_file_path: true
include_file_name: false
operators:
# Find out which format is used by kubernetes
- type: router
id: get-format
routes:
- output: parser-docker
expr: 'body matches "^\\{"'
- output: parser-crio
expr: 'body matches "^[^ Z]+ "'
- output: parser-containerd
expr: 'body matches "^[^ Z]+Z"'
# Parse CRI-O format
- type: regex_parser
id: parser-crio
regex: '^(?P<time>[^ Z]+) (?P<stream>stdout|stderr) (?P<logtag>[^ ]*) ?(?P<log>.*)$'
output: extract_metadata_from_filepath
timestamp:
parse_from: attributes.time
layout_type: gotime
layout: '2006-01-02T15:04:05.000000000-07:00'
# Parse CRI-Containerd format
- type: regex_parser
id: parser-containerd
regex: '^(?P<time>[^ ^Z]+Z) (?P<stream>stdout|stderr) (?P<logtag>[^ ]*) ?(?P<log>.*)$'
output: extract_metadata_from_filepath
timestamp:
parse_from: attributes.time
layout: '%Y-%m-%dT%H:%M:%S.%LZ'
# Parse Docker format
- type: json_parser
id: parser-docker
output: extract_metadata_from_filepath
timestamp:
parse_from: attributes.time
layout: '%Y-%m-%dT%H:%M:%S.%LZ'
# Extract metadata from file path
- type: regex_parser
id: extract_metadata_from_filepath
regex: '^.*\/(?P<namespace>[^_]+)_(?P<pod_name>[^_]+)_(?P<uid>[a-f0-9\-]+)\/(?P<container_name>[^\._]+)\/(?P<restart_count>\d+)\.log$'
parse_from: attributes["log.file.path"]
# Rename attributes
- type: move
from: attributes.stream
to: attributes["log.iostream"]
- type: move
from: attributes.container_name
to: attributes["k8s.container.name"]
- type: move
from: attributes.namespace
to: attributes["k8s.namespace.name"]
- type: move
from: attributes.pod_name
to: attributes["k8s.pod.name"]
- type: move
from: attributes.restart_count
to: attributes["k8s.container.restart_count"]
- type: move
from: attributes.uid
to: attributes["k8s.pod.uid"]
- type: move
from: attributes.log
to: body
this my configration
n

nitya-signoz

11/01/2022, 4:38 PM
I meant to exec into the otel collector k8s pod.
kubectl exec -it <pod_name> -n <namesapce> /bin/sh
and then
cd
into
/var/log/pods/
and check if you can see log files for any of the pod.
w

Wejdan

11/06/2022, 8:34 AM
cannot build pipelines: failed to create "otlp" exporter, in pipeline "logs": OTLP exporter config requires an Endpoint
this my new error
n

nitya-signoz

11/15/2022, 4:25 AM
can you check the otlp exporter configuration you will need to add the endpoint, something like this.