https://signoz.io logo
Title
r

Rade

02/10/2023, 10:27 AM
Hi all, can you please help me troubleshooting connection issue between two clusters?
[otel.javaagent 2023-02-10 10:19:31:968 +0000] [OkHttp <http://signoz.XXXXXXX:4317/...>] ERROR io.opentelemetry.exporter.internal.grpc.OkHttpGrpcExporter - Failed to export spans. The request could not be executed. Full error message: timeout
I am trying to send metrics from one GKE cluster to another GKE cluster where signoz is installed. Signoz cluster is fully opened to all traffic (both FW and Master node plane). Yet, I am still having this error message. All services in signoz cluster are ClusterIP type
s

Srikanth Chekuri

02/10/2023, 10:55 AM
@Prashant Shahi can probably help here
p

Prashant Shahi

02/10/2023, 10:58 AM
@Rade I don’t think it will be possible to communicate between clusters using ClusterIp type service. You would need to either use Ingress or switch the service type to NodePort or LoadBalancer.
Another approach to this would be using Istio. https://istio.io/v1.3/docs/examples/multicluster/gke/
r

Rade

02/10/2023, 11:04 AM
Which service should be switched to NodePort? All or just some specific? In both clusters?
@Prashant Shahi
p

Prashant Shahi

02/10/2023, 1:55 PM
@Rade OtelCollector service.