Hi everyone I followed this link to test django if can send

Question

Hi everyone I followed this link to test django if can send data to SigNoz <https signoz io blog opentelemetry django > I deployed SigNoz successfully and then I created this django app in kuberentes but got below errors `exec user process caused exec format error ` by using default image signoz sample django latest googled and found require to rebuild docker image after adding ` bin bash` in dockerfile and then I got below errors can you help why this issue happen I used offical dockerfile to build image ``` bin sh 1 opentelemetry instrument not found ```

Prashant Shahi · Answer

Hey < alan> That error usually happens when running on unsupported architecture could you please share information about the cluster nodes Do you have amd or arm nodes

alan · Answer

< Prashant Shahi> ```root uls tst01 gen1 7b64d94f9c 2zlr8 uname a Linux uls tst01 gen1 7b64d94f9c 2zlr8 5 4 0 1025 gkeop 26~18 04 1 Ubuntu SMP Mon Oct 4 03 14 34 UTC 2021 x86 64 x86 64 x86 64 GNU Linux root uls tst01 gen1 7b64d94f9c 2zlr8 ```

alan · Answer

I run docker command locally it is working but I got this error ` bin sh 1 opentelemetry instrument not found` when creating pod in kuberenetes so the docker image should be fine

Prashant Shahi · Answer

< alan> it looks like the SigNoz Django image is built for ARM I suppose you would have to build your own AMD Django image and use that

alan · Answer

< Prashant Shahi> my mac used 2 2 GHz 6 Core Intel Core i7 should not ARM

alan · Answer

this is not cpu architecture issue I rebuilt the image from kubernetes node but got same issue is this require some dependency to run opentelemetry instrument in k8s pls help < Prashant Shahi>

Prashant Shahi · Answer

< alan> let me build that on my machine and get back to you

Srikanth Chekuri · Answer

Also looking into it

Prashant Shahi · Answer

< alan> I was able to reproduce issue while building on both arm amd I was able to resolve it by passing the appropriate environments and complete commands ```docker run env OTEL METRICS EXPORTER=none env OTEL SERVICE NAME=djangoApp env OTEL EXPORTER OTLP ENDPOINT=http lt IP of SigNoz gt 4317 env DJANGO SETTINGS MODULE=mysite settings p 8000 8000 t signoz sample django latest opentelemetry instrument gunicorn mysite wsgi c gunicorn config py workers 2 threads 2 reload bind 0 0 0 0 8000``` Meanwhile I am working on improving the Dockerfile to fix this and reduce the image size Also I believe < Srikanth Chekuri> would be the best person who would have more knowledge on the Django sample app

alan · Answer

< Prashant Shahi> the dockerfile is fine I fixed that issue and i can run docker in my local without any issues but when I package it and run in kubernetes it threw the errors that ` bin sh 1 opentelemetry instrument not found` that s the point I wonder have you run this in kubernetes I am not able to instrument data to SigNoz

alan · Answer

< Srikanth Chekuri> can you help also thanks

Prashant Shahi · Answer

Can you share the k8s manifest for the sample django app

alan · Answer

alan · Answer

< Prashant Shahi>

Prashant Shahi · Answer

I see you are missing `command` Can you include the equipment environment variables and the command from the following in the manifest ```docker run env OTEL METRICS EXPORTER=none env OTEL SERVICE NAME=djangoApp env OTEL EXPORTER OTLP ENDPOINT=http lt IP of SigNoz gt 4317 env DJANGO SETTINGS MODULE=mysite settings p 8000 8000 t signoz sample django latest opentelemetry instrument gunicorn mysite wsgi c gunicorn config py workers 2 threads 2 reload bind 0 0 0 0 8000```

alan · Answer

same thing that s the reason I removed the command < Prashant Shahi>

Prashant Shahi · Answer

you would need to include it Also you mentioned that you had built amd image locally Can you point to that locally built image for k8s

alan · Answer

yes I did but it had same issue

alan · Answer

I can include and using image built from kubernetes node and try it again will let you know if any change this time

alan · Answer

``` op=Exists for 300s Events Type Reason Age From Message Normal Scheduled 2m44s default scheduler Successfully assigned alantest sample django deployment 799456b8cc 8vtr2 to uls tst01 gen1 7b64d94f9c p949m Normal Pulled 2m43s kubelet Successfully pulled image in 97 653019ms Normal Pulled 2m42s kubelet Successfully pulled image in 69 621653ms Normal Pulled 2m27s kubelet Successfully pulled image in 65 713606ms Normal Created 119s x4 over 2m43s kubelet Created container sample django Warning Failed 119s x4 over 2m43s kubelet Error failed to create containerd task OCI runtime create failed container linux go 380 starting container process caused exec opentelemetry instrument gunicorn mysite wsgi c gunicorn config py workers 2 threads 2 reload bind 0 0 0 0 8000 executable file not found in $PATH unknown Normal Pulled 119s kubelet Successfully pulled image in 65 166385ms Warning BackOff 84s x8 over 2m41s kubelet Back off restarting failed container Normal Pulling 73s x5 over 2m43s kubelet Pulling image ```

alan · Answer

this is the same I got previously and then I removed the command and args now I added back but issue persist

alan · Answer

< Prashant Shahi> < Srikanth Chekuri>

Prashant Shahi · Answer

replace command with this ```command opentelemetry instrument gunicorn mysite wsgi c gunicorn config py workers 2 threads 2 reload bind 0 0 0 0 8000 ```

alan · Answer

```gunicorn error unrecognized arguments none djangoApp <http my release signoz otel collector 4317> mysite settings ```

alan · Answer

< Prashant Shahi>

alan · Answer

no need argument

Prashant Shahi · Answer

yes can you remove the the `args`

alan · Answer

sure will try

alan · Answer

yes this working cool < Prashant Shahi>

Prashant Shahi · Answer

That s great

alan · Answer

but now issue is Signoz still not yet received the data from this django slightly smiling face

alan · Answer

```C02XGC2HJG5H apmtest ah1000259263$ k get svc n alantest NAME TYPE CLUSTER IP EXTERNAL IP PORT S AGE clickhouse operator metrics ClusterIP 172 30 240 102 lt none gt 8888 TCP 14d jaeger test agent ClusterIP 172 30 241 78 lt none gt 5775 UDP 6831 UDP 6832 UDP 5778 TCP 14271 TCP 10d jaeger test cassandra ClusterIP None lt none gt 7000 TCP 7001 TCP 7199 TCP 9042 TCP 9160 TCP 10d jaeger test collector ClusterIP 172 30 241 211 lt none gt 14250 TCP 14268 TCP 14269 TCP 10d jaeger test query ClusterIP 172 30 241 46 lt none gt 80 TCP 16687 TCP 10d my release clickhouse ClusterIP 172 30 240 62 lt none gt 8123 TCP 9000 TCP 14d my release signoz alertmanager ClusterIP 172 30 241 35 lt none gt 9093 TCP 14d my release signoz alertmanager headless ClusterIP None lt none gt 9093 TCP 14d my release signoz frontend ClusterIP 172 30 241 173 lt none gt 3301 TCP 14d my release signoz otel collector ClusterIP 172 30 240 254 lt none gt 4317 TCP 4318 TCP 55680 TCP 55681 TCP 14250 TCP 14268 TCP 9411 TCP 8888 TCP 8889 TCP 14d my release signoz otel collector metrics ClusterIP 172 30 241 20 lt none gt 4317 TCP 4318 TCP 55680 TCP 55681 TCP 14250 TCP 14268 TCP 9411 TCP 8888 TCP 14d my release signoz query service ClusterIP 172 30 240 200 lt none gt 8080 TCP 14d my release zookeeper ClusterIP 172 30 240 207 lt none gt 2181 TCP 2888 TCP 3888 TCP 14d my release zookeeper headless ClusterIP None lt none gt 2181 TCP 2888 TCP 3888 TCP```

alan · Answer

I set OTEL exporter endpoint as below in deployment is this correct < Prashant Shahi> ``` name OTEL EXPORTER OTLP ENDPOINT value ```

Prashant Shahi · Answer

< alan> As mentioned in the blog you would have to create poll and visit polls Steps 1 Browsing the app and checking data with SigNoz 2 a Visit and create a question for poll 3 b Then visit the list of polls at and explore the polls 4 c The data should be visible now in SigNoz at `http lt IP of SigNoz gt 3301`

Prashant Shahi · Answer

If you haven t updated Dockerfile admin user `ankitnayan` password `password`

alan · Answer

yes will try and then back to you

alan · Answer

I suspect some issue with this ` bind 0 0 0 0 8000 ` in yaml when I create virtual service I got `400` error < Prashant Shahi>

alan · Answer

if so what else I need to change I am not familiar with django so asking < Prashant Shahi>

alan · Answer

this bind to <http 0 0 0 0 8000> is this correct

alan · Answer

< Prashant Shahi>

alan · Answer

< Srikanth Chekuri>

Prashant Shahi · Answer

can you port forward and check if it works in localhost

Prashant Shahi · Answer

Try the following endpoints ``` ```

alan · Answer

ok port forward is good

alan · Answer

and I create poll I can see logs get updated but signiz didn t get updated

alan · Answer

< Prashant Shahi>

Prashant Shahi · Answer

visit polls vote

Prashant Shahi · Answer

check SigNoz

Prashant Shahi · Answer

it should show up

Prashant Shahi · Answer

check SigNoz UI

Prashant Shahi · Answer

hit refresh

Prashant Shahi · Answer

also check otel collector logs

Prashant Shahi · Answer

first of all you should check if the OtelCollector Otlp endpoint is accessible Follow the troubleshooting guide for k8s <https signoz io docs install troubleshooting >

Prashant Shahi · Answer

```kubectl n platform run troubleshoot image=signoz troubleshoot restart= OnFailure i tty rm command troubleshoot checkEndpoint endpoint=my release signoz otel collector platform svc cluster local 4317```

alan · Answer

will check

alan · Answer

```C02XGC2HJG5H jaeger ah1000259263$ kubectl n alantest run troubleshoot image=<http artifactory wdc com 6609 signoz troubleshoot|artifactory wdc com 6609 signoz troubleshoot> l e2 criticality=2 e2 environment=DEV e2 owner=E2 Owner e2 project=e2 migration e2 support contact=E2 Support gt restart= OnFailure i tty rm command troubleshoot checkEndpoint gt endpoint=my release signoz otel collector 4317 If you don t see a command prompt try pressing enter Error attaching falling back to logs unable to upgrade connection container troubleshoot not found in pod troubleshoot alantest 2022 04 28T09 10 37 205Z INFO troubleshoot main go 28 STARTING 2022 04 28T09 10 37 208Z INFO checkEndpoint checkEndpoint go 41 checking reachability of SigNoz endpoint 2022 04 28T09 10 37 335Z INFO troubleshoot main go 46 Successfully sent sample data to signoz pod troubleshoot deleted```

alan · Answer

< Prashant Shahi> this means OtelCollector Otlp endpoint is accessible correct

alan · Answer

`troubleshoot main go 46 Successfully sent sample data to signoz`

Prashant Shahi · Answer

yes that means SigNoz cluster is up and running properly

alan · Answer

then why signiz no data

alan · Answer

let me know anything you need this is log from my release signoz otel collector 7d895854c9 b7srl

alan · Answer

1 now the django is running successfully 2 run the troubleshoot pod tested my release signoz otel collector 4317 also good 3 add few polls in django 4 but signiz still shows no data

alan · Answer

< Prashant Shahi> let me know anything you need for troubleshooting

alan · Answer

is http need to be removed ``` name OTEL EXPORTER OTLP ENDPOINT value <http my release signoz otel collector 4317>```

Prashant Shahi · Answer

it all looks good to me can you create another poll and vote again

Prashant Shahi · Answer

< alan>

alan · Answer

sure

alan · Answer

checked but still no data

Prashant Shahi · Answer

http is optional in some SDK for grpc port

alan · Answer

nothing changed in my release signoz otel collector 7d895854c9 b7srl

alan · Answer

smiling face with tear

alan · Answer

< Prashant Shahi> not sure if this conflicts I installed a Instrumentation by following this thread