Hi all, can you please help me troubleshooting connection issue between two clusters?

[otel.javaagent 2023-02-10 10:19:31:968 +0000] [OkHttp <http://signoz.XXXXXXX:4317/...>] ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: timeout

I am trying to send metrics from one GKE cluster to another GKE cluster where signoz is installed. Signoz cluster is fully opened to all traffic (both FW and Master node plane). Yet, I am still having this error message. All services in signoz cluster are ClusterIP type

@Prashant Shahi can probably help here

@Rade I don’t think it will be possible to communicate between clusters using ClusterIp type service. You would need to either use Ingress or switch the service type to NodePort or LoadBalancer.

Which service should be switched to NodePort? All or just some specific? In both clusters?

@Rade OtelCollector service.