I m using the docker compose clickhouse setup There is an er

Question

I m using the docker compose clickhouse setup There is an error in the logs every 5 seconds ```otel collector 1 | 2022 03 22T08 51 28 780Z info service collector go 124 Everything is ready Begin running and processing data otel collector 1 | time= 2022 03 22T08 51 28Z level=error msg= invalid character x06 in string literal component=clickhouse otel collector 1 | time= 2022 03 22T08 51 33Z level=error msg= invalid character x06 in string literal component=clickhouse otel collector 1 | time= 2022 03 22T08 51 38Z level=error msg= invalid character x06 in string literal component=clickhouse otel collector 1 | time= 2022 03 22T08 51 43Z level=error msg= invalid character x06 in string literal component=clickhouse otel collector 1 | time= 2022 03 22T08 51 48Z level=error msg= invalid character x06 in string literal component=clickhouse``` How can I debug this Or can I ignore this Unfortunately the app is behaving strange the trace page is working but if I click on a trace the page is blank

Ankit Nayan · Answer

< U036R4SAG0H> let us check this out Is this some sample application application Any way to replicate sending the data You can use console exporter in opentelemetry which outputs the data to terminal and we can check what went wrong from that data

Ankit Nayan · Answer

also which language is the application in

Uwe Voelker · Answer

It is not the demo app It is in JavaScript

Uwe Voelker · Answer

I did use the ConsoleSpanExporter and it looked innocent

Uwe Voelker · Answer

Why does it happen every 5 seconds

Uwe Voelker · Answer

The trace detail page seems to work in the beginning with a small dataset and later stops working blank page

Uwe Voelker · Answer

I just reset all data

Ankit Nayan · Answer

trace detail page breaks due to broken span data probably due to the otel collector error you see in logs

Ankit Nayan · Answer

can you paste the output of `ConsoleSpanExporter` here will try to check if anything is wrong there

Uwe Voelker · Answer

After data reset the error comes immediately but I also ingest all the time But the trace detail page works in the beginning

Uwe Voelker · Answer

I will setup console export again and post it here in a few hours have to leave now

Uwe Voelker · Answer

Is there a way to have both exporters at the same time like Linux `tee`

Ankit Nayan · Answer

yes there should be a way to do that Checkout opentelemetry js docs to add an exporter to a provider and try adding multiple exporters I haven t tried it yet so cannot be sure about it

Uwe Voelker · Answer

mmh since about 3 am in the morning it stopped accepting traces and now I m seeing the following error ```otel collector 1 | time= 2022 03 23T16 02 43Z level=error msg= invalid character x06 in string literal component=clickhouse otel collector 1 | 2022 03 23T16 02 45 592Z error signozspanmetricsprocessor processor go 412 value not found in metricKeyToDimensions cache by key xxx app x00STATUS CODE ERROR x00429 x00GET kind processor name signozspanmetrics prometheus otel collector 1 | otel collector 1 | home codegeas code work opentelemetry collector contrib processor signozspanmetricsprocessor processor go 412 otel collector 1 | otel collector 1 | home codegeas code work opentelemetry collector contrib processor signozspanmetricsprocessor processor go 314 otel collector 1 | otel collector 1 | home codegeas code work opentelemetry collector contrib processor signozspanmetricsprocessor processor go 283 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector v0 43 0 receiver otlpreceiver internal trace otlp go 60 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector model v0 44 0 otlpgrpc traces go 166 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector model v0 44 0 internal data protogen collector trace v1 trace service pb go 217 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector v0 43 0 config configgrpc configgrpc go 382 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1116 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io contrib instrumentation google golang org grpc otelgrpc v0 28 0 interceptor go 325 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1119 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1121 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector model v0 44 0 internal data protogen collector trace v1 trace service pb go 219 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1282 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1616 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 921 otel collector 1 | 2022 03 23T16 02 45 640Z error signozspanmetricsprocessor processor go 412 value not found in metricKeyToDimensions cache by key xxx app x00STATUS CODE ERROR x00429 x00GET kind processor name signozspanmetrics prometheus otel collector 1 | otel collector 1 | home codegeas code work opentelemetry collector contrib processor signozspanmetricsprocessor processor go 412 otel collector 1 | otel collector 1 | home codegeas code work opentelemetry collector contrib processor signozspanmetricsprocessor processor go 314 otel collector 1 | otel collector 1 | home codegeas code work opentelemetry collector contrib processor signozspanmetricsprocessor processor go 283 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector v0 43 0 receiver otlpreceiver internal trace otlp go 60 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector model v0 44 0 otlpgrpc traces go 166 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector model v0 44 0 internal data protogen collector trace v1 trace service pb go 217 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector v0 43 0 config configgrpc configgrpc go 382 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1116 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io contrib instrumentation google golang org grpc otelgrpc v0 28 0 interceptor go 325 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1119 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1121 otel collector 1 | otel collector 1 | home codegeas go pkg mod go opentelemetry io collector model v0 44 0 internal data protogen collector trace v1 trace service pb go 219 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1282 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 1616 otel collector 1 | otel collector 1 | home codegeas go pkg mod google golang org grpc v1 44 0 server go 921```