We do it as a sidecar since we use ECS with Fargate ^^ but same logic should play well with k8s as well

Hi Pranay

image.png

how to populate trace_id in log details.. trace id coming as part of body

I tried this way..

processors: attributes/log: actions: - key: trace_id action: upsert from_attribute: log.body.req.trace_id

logs: receivers: [otlp, tcplog/docker] processors: [attributes/log, batch] exporters: [clickhouselogsexporter]

but it’s not working

so we were able to extract it from the log

Hi Pranay, FYI. I managed to populate trace_id and span_id from my log body with below processor configurations in otel-collector-config.yaml file

logstransform/json_parser: operators: - type: json_parser parse_from: body parse_to: attributes logstransform/trace: operators: - type: trace_parser if: ‘“trace_id” in attributes or “span_id” in attributes’ trace_id: parse_from: attributes.trace_id span_id: parse_from: attributes.span_id output: remove_trace_id - id: remove_trace_id type: remove if: ‘“trace_id” in attributes’ field: attributes.trace_id output: remove_span_id - id: remove_span_id type: remove if: ‘“span_id” in attributes’ field: attributes.span_id

no need to create pipelines

‘use strict’; import { OTLPTraceExporter } from ‘@opentelemetry/exporter-trace-otlp-http’; import { HttpInstrumentation } from ‘@opentelemetry/instrumentation-http’; import { NestInstrumentation } from ‘@opentelemetry/instrumentation-nestjs-core’; import { PinoInstrumentation } from ‘@opentelemetry/instrumentation-pino’; import { ExpressInstrumentation } from ‘@opentelemetry/instrumentation-express’; import { PrismaInstrumentation } from ‘@prisma/instrumentation’; import { PgInstrumentation } from ‘@opentelemetry/instrumentation-pg’; import { Resource } from ‘@opentelemetry/resources’; import * as opentelemetry from ‘@opentelemetry/sdk-node’; import { SimpleSpanProcessor, ConsoleSpanExporter } from ‘@opentelemetry/sdk-trace-base’; import { SEMRESATTRS_SERVICE_NAMESPACE, SEMRESATTRS_SERVICE_NAME } from ‘@opentelemetry/semantic-conventions’; import { getNodeAutoInstrumentations } from ‘@opentelemetry/auto-instrumentations-node’; // // Enable logging for debugging // import { diag, DiagConsoleLogger, DiagLogLevel } from ‘@opentelemetry/api’; // diag.setLogger(new DiagConsoleLogger(), DiagLogLevel.DEBUG); // Configure the SDK to export telemetry data to the console // Enable all auto-instrumentations from the meta package const exporterOptions = { url: ‘http://localhost:4318/v1/traces’, }; const traceExporter = new OTLPTraceExporter(exporterOptions); const sdk = new opentelemetry.NodeSDK({ traceExporter, instrumentations: [ // getNodeAutoInstrumentations(), new NestInstrumentation(), new HttpInstrumentation(), // new PinoInstrumentation(), new ExpressInstrumentation(), new PgInstrumentation(), new PrismaInstrumentation({ middleware: true }) ], resource: new Resource({ [SEMRESATTRS_SERVICE_NAMESPACE]: ‘Testnet’, [SEMRESATTRS_SERVICE_NAME]: ‘Portal’ }), }); // initialize the SDK and register with the OpenTelemetry API // this enables the API to record telemetry sdk.start(); // gracefully shut down the SDK on process exit process.on(‘SIGTERM’, () => { sdk .shutdown() .then(() => console.log(‘Tracing terminated’)) .catch((error) => console.log(‘Error terminating tracing’, error)) .finally(() => process.exit(0)); }); export default sdk;

image.png,image.png