https://signoz.io logo
#general
Title
# general
b

Brian Richardson

07/21/2022, 1:15 PM
Copy code
builder.Services.AddOpenTelemetryTracing(tracerProviderBuilder =>
{
    tracerProviderBuilder
        .AddSource(serviceName)
        .SetResourceBuilder(
            ResourceBuilder.CreateDefault()
                .AddService(serviceName: serviceName, serviceVersion: serviceVersion))
        .AddOtlpExporter(otlp =>
        {
            otlp.Protocol = OtlpExportProtocol.Grpc;
            otlp.Endpoint = new Uri(builder.Configuration["Telemetry:CollectorUrl"]);
        })
        .AddHttpClientInstrumentation()
        .AddRedisInstrumentation()
        .AddAspNetCoreInstrumentation()
        .AddMassTransitInstrumentation();
});
s

Srikanth Chekuri

07/21/2022, 2:05 PM
You will probably have to enable unencrypted client something like this https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/examples/Console/TestOtlpExporter.cs#L61
b

Brian Richardson

07/21/2022, 3:30 PM
hmmm. that didn't seem to have any effect. I note, too, that the console exporter doesn't show anything. Should I not be expecting the raw requests to show up in the telemetry?
s

Srikanth Chekuri

07/21/2022, 3:31 PM
If you are not seeing anything in the console exporter as well then there is an issue with pipeline setup.
b

Brian Richardson

07/21/2022, 3:32 PM
ok... but there's nothing different here than the backend microservices, which are correctly sending their telemetry
from the collector logs:
Copy code
2022-07-21T13:19:39.262Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "clickhouse: dateTime overflow. timestamp must be between 1925-01-01 00:00:00 and 2283-11-11 00:00:00"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
s

Srikanth Chekuri

07/21/2022, 4:33 PM
That's strange. Can you get the console exporter working? How does the data look like for values using that?
b

Brian Richardson

07/21/2022, 4:34 PM
do you know how I can capture the data going to the collector?
I have tried purging the volume to see if there's just something weird stuck in the queue
s

Srikanth Chekuri

07/21/2022, 4:36 PM
I am aware of such hooks. If you want to see the data I think the best option is to use console exporter.
b

Brian Richardson

07/21/2022, 4:36 PM
ok
but it never showed the spans it was trying to send...
when I purged the volumes, the error went away, but the data is still not showing up in either exporter
can I attach all logs here for someone to look at?
s

Srikanth Chekuri

07/21/2022, 4:38 PM
I want to understand if the issue with the collector or your app instrumented with SDK. First step is to verify application produces telemetry with console exporter.
b

Brian Richardson

07/21/2022, 4:38 PM
yeah, I can understand that
but it should be working out of the box with the C# DI code I posted, shouldn't it?
s

Srikanth Chekuri

07/21/2022, 4:46 PM
i don't remember the exact setting but otel dotnet sdk has setting to enable to diagnostic logging,, did you try that
b

Brian Richardson

07/21/2022, 4:46 PM
I will see if I can find how to do that
Copy code
otel-collector_1          | Error: cannot build exporters: error creating clickhousetraces exporter: error connecting to primary db: dial tcp: lookup clickhouse on 127.0.0.11:53: no such host
otel-collector_1          | 2022/07/21 17:14:38 collector server run finished with error: cannot build exporters: error creating clickhousetraces exporter: error connecting to primary db: dial tcp: lookup clickhouse on 127.0.0.11:53: no such host
hmmm that looks like a good place to start
s

Srikanth Chekuri

07/21/2022, 5:24 PM
How are you deploying the signoz?
b

Brian Richardson

07/21/2022, 5:24 PM
as per the docs:
docker-compose -f docker/clickhouse-setup/docker-compose.yaml
I modified the compose to use an external network, but that's it
s

Srikanth Chekuri

07/21/2022, 5:25 PM
It's unable to connect to clickhouse db
b

Brian Richardson

07/21/2022, 5:25 PM
btw, the hostname lookup error was just a typo on my part. I've fixed it and I'm back to square one. no errors but no console export either.
dumb question maybe, but .NET 6 is supported?
s

Srikanth Chekuri

07/21/2022, 5:36 PM
Yes
b

Brian Richardson

07/21/2022, 7:11 PM
ok it is showing in the console log now. It looks like
AddRedisInstrumentation()
was throwing an exception. Removing it has logged the activities to the console.
but it is still not making it to the collector
I have tried both Jaeger and Signoz collectors, and Signoz using both HttpProtobuf and gRPC (and the appropriate port)
I can see no sign that any attempt is being made to send the data...
s

Srikanth Chekuri

07/21/2022, 7:24 PM
Can you share the endpoint you are setting?
b

Brian Richardson

07/21/2022, 7:24 PM
sorry, I don't quite understand
you mean the value of Telemetry:CollectorUrl?
I have been using http://otel-collector:4317 (for gRPC) or :4318 (for HttpProtobuf)
I've connected to the web api and verified that otel-collector resolves correctly
s

Srikanth Chekuri

07/21/2022, 7:38 PM
Where is you testing app running? Wouldn't the url be
<http://localhost:4317>
or
<http://localhost:4318>
?
b

Brian Richardson

07/21/2022, 7:38 PM
no, it is running in docker container
and collector is running in a different container on same network
s

Srikanth Chekuri

07/21/2022, 7:39 PM
So you are seeing the spans on console but not when you OTLP exporter?
b

Brian Richardson

07/21/2022, 7:41 PM
that's correct
but I guess that's not a Signoz problem
s

Srikanth Chekuri

07/21/2022, 7:42 PM
Yes, can you run a very basic app locally and send data at
<http://localhost:4317>
and see if it is successful?
b

Brian Richardson

07/21/2022, 7:43 PM
I'm sure it does
however, I think I've taken up enough of your time
s

Srikanth Chekuri

07/21/2022, 7:45 PM
Haha It's hard to debug remotely. I suspect there is something wrong with OTLP setup in the app.
b

Brian Richardson

07/21/2022, 7:46 PM
well, as I have said, I am using the exact same setup in the worker service as the web api, and the worker service logs traces and the web api does not
but, it is an OLTP issue and I won't bother you with it any further. I only avoid making a "test" application on localhost because it proves nothing and doesn't really enlighten me as to the cause
I was only looking for some insight as to what might be wrong with the DI declaration (solved), but now we end up with a different problem. As you say, you cannot debug remotely.
5 Views