Hello Team, Could you please assist me with an issue I'm facing? I have installed and configured Sig...
p
Hello Team, Could you please assist me with an issue I'm facing? I have installed and configured SigNoz on an AWS EKS cluster. It is working as expected with the OTEL collector endpoint pointing to the default service/pod endpoint like this:
Copy code
otlp:
        endpoint: <http://signoz-otel-collector.platform.svc.cluster.local:4317>
Our team has requested to externalize this endpoint using Ingress, so we can use the same endpoint across different clusters instead of configuring OTEL in each cluster. We have installed the AWS Load Balancer Controller to use as the ingress controller. I created an Ingress resource using the following specification referenced in the Helm values file:
Copy code
otelCollector:
  ingress:
    # -- Enable ingress for otelCollector
    enabled: true
    # -- Annotations to otelCollector Ingress
    annotations:
      alb.ingress.kubernetes.io/load-balancer-name: signoz-collector-lb
      kubernetes.io/ingress.class: alb
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80}, {"HTTPS":443}]'
      alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:XXXXXXXXX:certificate/XXXXXXXXXXX
      alb.ingress.kubernetes.io/ip-address-type: ipv4
      alb.ingress.kubernetes.io/target-type: ip
    # -- Frontend Ingress Host names with their path details
    hosts:
      - host: <HOST_URL>
        paths:
          - path: /
            pathType: ImplementationSpecific
            port: 4318
I was able to create the Load Balancer, but it is failing the health check, which prevents access to the endpoint from outside. Can someone please check and help me resolve this issue? I am using EKS version 1.27 and the Helm chart version is "signoz-0.44.0".
n
@Prashant Shahi can you help here
p
@Prashant Shahi Once we have health check passed then only it will route traffic to backend pods, I see It is not passing health check on port 4318 due to that I am not able to access load balancer endpoint from outside.
p
@Prathap ch can you include the following?
Copy code
otelCollector:
  ports:
    health-check:
      enabled: true
      containerPort: 13133
      servicePort: 13133
      protocol: TCP
p
@Prashant Shahi Here issue is with target group registration to the pod, since it is unhealthy as getting 404 Notfound with below ingress rule. I tried same what you suggested but stilll same error. While registering it is looking for path as ''/' with port 4318 where we have http pod service is listening, not sure do we have any other path that I can access for OTEL collector pod to get 200 response.
Copy code
hosts:
      - host: <http://otelcollector.domain.com|otelcollector.domain.com>
        paths:
          - path: /
            pathType: ImplementationSpecific
            port: 4318
@Prashant Shahi I updated same port number in ingress rule as well, now I see it is healthy and able to access otel collector url from outside, I hope this is right response..
Copy code
{
  "status": "Server available",
  "upSince": "2024-07-29T21:09:23.292708911Z",
  "uptime": "17m31.865224939s"
}

###Ingress Configuration#####
    hosts:
      - host: <HOST_URL>
        paths:
          - path: /
            pathType: ImplementationSpecific
            port: 13133
@Prashant Shahi I see 200 response, but when I tried to use this domain as OTEL Endpoint I am seeing 404, does this 13133 ports really accepts http/https requests?
g
@Prathap ch im also facing the same any work around done