https://signoz.io logo
#support
Title
# support
u

Ulisses Silva

08/28/2023, 1:02 PM
Hello, i'm having the following problem: I started to test signoz and have setup 3 java applications that are using the otel java agent on tomcat to send data directly to signoz collector on port 4317. Sometimes i need to restart the signoz server and one of the applications just can't send data until i do a a restart of this application.
[otel.javaagent 2023-08-28 08:09:36:463 -0400] [OkHttp <http://signoz.server:4317/...>] ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180
[otel.javaagent 2023-08-28 08:09:41:467 -0400] [OkHttp <http://signoz.server:4317/...>] ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180
[otel.javaagent 2023-08-28 08:09:42:582 -0400] [OkHttp <http://signoz.server:4317/...>] ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export logs. The request could not be executed. Full error message: FRAME_SIZE_ERROR: 4740180
There is any option on otel agent or signoz i'm missing?
s

Srikanth Chekuri

08/28/2023, 1:06 PM
u

Ulisses Silva

08/28/2023, 1:46 PM
I have updated the otel-collector-config.yaml with receivers.otlp.protocols.grpc.max_recv_msg_size_mib: 100, restarted the containers with docker compose down/up, but the problem continues.
To add some context, all 3 applications are running in the same VM server, but on different tomcat instances.
s

Srikanth Chekuri

08/29/2023, 3:21 AM
Lower the agent batch size to export most frequently and check.
u

Ulisses Silva

08/29/2023, 6:52 PM
otel.bsp.max.export.batch.size or otel.blrp.max.export.batch.size? or both?
s

Srikanth Chekuri

08/30/2023, 12:11 PM
You might have to tweak both. How many spans does your app roughly produce? You need to adjust the settings, or you could drop the spans with head-based sampling if the volume and size are big.
u

Ulisses Silva

08/30/2023, 12:42 PM
This app is generating like 2220 spans per minute. There are some operations that generates 28000+ spans. I can't even open the Trace Detail page because the browser can't handle that number of elements and will freeze.
4 Views