https://signoz.io
Join Slack
Hi, there's a difference docs about the backend port. One doc says it's `4318` and the grid says `43...
# support
a
Hi, there's a difference docs about the backend port. One doc says it's 
4318
and the grid says 
4317
Can anyone please tell me which one of the pod is the backend pod? Is it the 
otel-collector-5c449d4568-ctwqv
? We're using GKE, and managed ALB, I have exposed the frontend pod and it's accessible via internet but it's a bit confusing which one is the backend.
Copy code
kgp -n platform -o wide
NAME                                                        READY   STATUS    RESTARTS        AGE     IP            
chi-signoz-release-clickhouse-cluster-0-0-0                 1/1     Running   1 (4d18h ago)   4d18h   10.36.0.20 
signoz-release-alertmanager-0                               1/1     Running   0               4d18h   10.36.2.31 
signoz-release-clickhouse-operator-cbc86677c-cvvvl          2/2     Running   0               4d18h   10.36.2.9  
signoz-release-frontend-7bc74bb948-zjpsb                    1/1     Running   0               4d18h   10.36.2.8  
signoz-release-k8s-infra-otel-agent-56wwc                   1/1     Running   0               11h     10.36.3.3  
signoz-release-k8s-infra-otel-agent-b2jsk                   1/1     Running   0               4d18h   10.36.2.2  
signoz-release-k8s-infra-otel-agent-b6znv                   1/1     Running   0               4d18h   10.36.0.2  
signoz-release-k8s-infra-otel-agent-kvttd                   1/1     Running   0               4d18h   10.36.5.2  
signoz-release-k8s-infra-otel-agent-lc6z9                   1/1     Running   0               4d18h   10.36.9.2  
signoz-release-k8s-infra-otel-agent-mm7qt                   1/1     Running   0               21h     10.36.4.3  
signoz-release-k8s-infra-otel-deployment-767fc8b954-fhnkd   1/1     Running   0               4d18h   10.36.2.30 
signoz-release-otel-collector-5c449d4568-ctwqv              1/1     Running   2 (4d18h ago)   4d18h   10.36.5.10 
signoz-release-otel-collector-metrics-6b9985fb8-z6dzw       1/1     Running   0               4d18h   10.36.5.13 
signoz-release-query-service-0                              1/1     Running   0               111m    10.36.3.121
signoz-release-zookeeper-0                                  1/1     Running   0               4d18h   10.36.2.32
Copy code
kgs -n platform -o wide
NAME                                         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)                                                            
chi-signoz-release-clickhouse-cluster-0-0    ClusterIP   None           <none>        9000/TCP,8123/TCP,9009/TCP                                         
signoz-release-alertmanager                  ClusterIP   10.40.8.58     <none>        9093/TCP                                                           
signoz-release-alertmanager-headless         ClusterIP   None           <none>        9093/TCP                                                           
signoz-release-clickhouse                    ClusterIP   10.40.4.184    <none>        8123/TCP,9000/TCP                                                  
signoz-release-clickhouse-operator-metrics   ClusterIP   10.40.7.45     <none>        8888/TCP                                                           
signoz-release-frontend                      ClusterIP   10.40.9.105    <none>        3301/TCP                                                           
signoz-release-k8s-infra-otel-agent          ClusterIP   10.40.3.42     <none>        13133/TCP,8888/TCP,4317/TCP,4318/TCP                               
signoz-release-k8s-infra-otel-deployment     ClusterIP   10.40.9.148    <none>        13133/TCP                                                          
signoz-release-otel-collector                ClusterIP   10.40.15.206   <none>        14250/TCP,14268/TCP,8081/TCP,8082/TCP,8888/TCP,4317/TCP,4318/TCP   
signoz-release-otel-collector-metrics        ClusterIP   10.40.6.68     <none>        13133/TCP                                                          
signoz-release-query-service                 ClusterIP   10.40.13.83    <none>        8080/TCP,8085/TCP,4320/TCP                                         
signoz-release-zookeeper                     ClusterIP   10.40.4.138    <none>        2181/TCP,2888/TCP,3888/TCP                                         
signoz-release-zookeeper-headless            ClusterIP   None           <none>        2181/TCP,2888/TCP,3888/TCP
k
query-service is the backend pod for the frontend pod otel-collector is the backend pod for your application service which is being instrumented using otel
p
Ports 4318 and 4317 are the data ingestion ports for the opentelemetry collectors. 4318 is accessed with the HTTP protocol and 4317 with gRPC. You should be able to see something like
Copy code
# Data sources: traces, metrics, logs
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
      http:
        endpoint: 0.0.0.0:4318
somewhere in your config.
šŸ‘ 1
You don't really communicate with Otel. As stated before, you're looking for query-service most likely. If you're looking to query the data, I would even suggest interacting with Clickhouse directly.
a
@Pedro Carvalho I don't want to query the data. I want to ingest the data from our applications. I'm seeing healthcheck fails for the port 4318 in GCP. Is there a specific endpoint for checking health check for ingestion backend?
p
Healthcheck for opentelemetry collector is usually on port 13133, which I can actually see that you don't have on the service above. You can enable it with
Copy code
ports:
    health-check:
      containerPort: 13133
      enabled: true
      protocol: TCP
      servicePort: 13133
With that said, if you're trying to see if the thing is running, I might have a different suggetion. Port forward 4318 on that pod and just access it on the browser. You should see this a 404 error like this image. That means the collector is running properly and the HTTP receiver is listening to that port. If it's not then there's a different underlying issue with the otel config.
a
I'll disable health check, the ALB won't forward requests if the health checks are failing.
p
Oh right healthcheck is an extension and also needs to be under 
service.extensions
Is the pod there and live tho?
a
Yes, I'm able to access the app via UI and see logs that it automatically collects from k8s
p
It's probably collecting the logs from k8s through the 
k8s-infra
collector tho, so not necessarily from the one you're trying to target.
(I'm not associated with signoz btw)
Either way, find the pod under 
signoz-release-otel-collector
service's selector, and check its status/logs. If nothing is off, port forward the http receiver port and if it behaves normally. Then if data doesn't show up when sent to it, there might be another issue and next step would be looking at the config values on helm OR at the configuration of the exporters sending to it.
a
Can anyone from the Signoz team help here? @Chitransh Gupta @Vikrant Gupta
I forwarded the port 
kubectl port-forward signoz-release-otel-collector-5c449d4568-ctwqv 4318:4318 -n platform
and it's showing 
405
so I guess something is working? @Pedro Carvalho Now I somehow need to make the health check return status 200 so the ALB can forward requests, I can't disable health check in GCP
v
@Prashant Shahi @Vibhu Pandey Can you please help here . Thanks!
a
@Vikrant Gupta @Prashant Shahi @Vibhu Pandey @Pranay waiting for help
v
Hi @Anurag Bhagsain, Looking at your browser request, you're trying to do a GET call on a post endpoint. All endpoints are POST endpoints which take the telemetry data as the payload.
Couple of things here: ā€¢ Your frontend pod with port 3301 needs to be exposed via a load balancer (ALB in your case) so that you can open the signoz UI. That's where you will be able to configure dashboards, alerts, build queries, see the service map, see a list of services and basically be able to play with all of your telemetry data.
ā€¢ Now the second part would be to actually start sending telemetry data to signoz so that you can actually start getting values in the UI you exposed in the previous step. That's where the otel collector comes in. (
otel-collector-5c449d4568-ctwqv)
. In order to send telemetry data from your applications written in Go, Java, C, Python and so on, you need to instrument them using its corresponding opentelemetry library and then point it to your 
otel-collector-5c449d4568-ctwqv
host:4317 or the same host:4318.
a
thanks @Vibhu Pandey but we decided to ahead with Elastic co for now, if cost becomes an issue we'd prob try Signoz again
5 Views
Open in Slack
PreviousNext
Powered by

SigNoz Community

SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.

Powered by