i tried to connect the logs in kubernetes but i faced this error no files match the configured include patterns {"kind": "receiver", "name": "filelog/k8s", "pipeline": "logs", "component": "fileconsumer", "include": ["/var/log/pods/*/*/*.log"], "exclude": []} could anyone help me?

Can you give us more info about your kubernetes deployment like you are using EKS or something like minikube ?

no, its oracle cloud and it's connected with open telemetry

Hi, we can you please exec into the OTEL collector pod and check log files are present inside

/var/log/pods/<pod_name>/<container_name>/*.log

also check the permissions of the log file. We use the root user to read these files so might be some issue with that.

filelog/k8s:

include:

# Include logs from all container

- /var/log/pods/<pod_name>/<container_name>/*.log

exclude:

# Exclude logs from all containers from kube-system namespace

- /var/log/pods/kube-system_*/*/*.log

# Exclude logs from all hotrod containers

- /var/log/pods/*_hotrod-*/*/*.log

- /var/log/pods/*_locust-*_*/*/*.log

start_at: beginning

include_file_path: true

include_file_name: false

operators:

# Find out which format is used by kubernetes

- type: router

id: get-format

routes:

- output: parser-docker

expr: 'body matches "^\\{"'

- output: parser-crio

expr: 'body matches "^[^ Z]+ "'

- output: parser-containerd

expr: 'body matches "^[^ Z]+Z"'

# Parse CRI-O format

- type: regex_parser

id: parser-crio

regex: '^(?P<time>[^ Z]+) (?P<stream>stdout|stderr) (?P<logtag>[^ ]*) ?(?P<log>.*)$'

output: extract_metadata_from_filepath

timestamp:

parse_from: attributes.time

layout_type: gotime

layout: '2006-01-02T15:04:05.000000000-07:00'

# Parse CRI-Containerd format

- type: regex_parser

id: parser-containerd

regex: '^(?P<time>[^ ^Z]+Z) (?P<stream>stdout|stderr) (?P<logtag>[^ ]*) ?(?P<log>.*)$'

output: extract_metadata_from_filepath

timestamp:

parse_from: attributes.time

layout: '%Y-%m-%dT%H:%M:%S.%LZ'

# Parse Docker format

- type: json_parser

id: parser-docker

output: extract_metadata_from_filepath

timestamp:

parse_from: attributes.time

layout: '%Y-%m-%dT%H:%M:%S.%LZ'

# Extract metadata from file path

- type: regex_parser

id: extract_metadata_from_filepath

regex: '^.*\/(?P<namespace>[^_]+)_(?P<pod_name>[^_]+)_(?P<uid>[a-f0-9\-]+)\/(?P<container_name>[^\._]+)\/(?P<restart_count>\d+)\.log$'

parse_from: attributes["log.file.path"]

# Rename attributes

- type: move

from: attributes.stream

to: attributes["log.iostream"]

- type: move

from: attributes.container_name

to: attributes["k8s.container.name"]

- type: move

from: attributes.namespace

to: attributes["k8s.namespace.name"]

- type: move

from: attributes.pod_name

to: attributes["k8s.pod.name"]

- type: move

from: attributes.restart_count

to: attributes["k8s.container.restart_count"]

- type: move

from: attributes.uid

to: attributes["k8s.pod.uid"]

- type: move

from: attributes.log

to: body

I meant to exec into the otel collector k8s pod.

kubectl exec -it <pod_name> -n <namesapce> /bin/sh

and then

cd

into

/var/log/pods/

and check if you can see log files for any of the pod.

cannot build pipelines: failed to create "otlp" exporter, in pipeline "logs": OTLP exporter config requires an Endpoint

can you check the otlp exporter configuration you will need to add the endpoint, something like this.