Title
j
Jose Infazon
02/17/2023, 11:50 AMHi Everyone - I've deployed signoz on an AWS EC2 instance. This instance is scheduled to start and shutdown at certain hours. When it starts, all docker signoz containers start (I've set restart: unless stopped" in the docker compose. But I've an issue with the front-end service and I can't login, I always need to restart front-end service. Is there any dependency I need to add?
v
Vishal Sharma
02/17/2023, 1:26 PM@Prashant Shahi Can you please help him here?
p
Prashant Shahi
02/17/2023, 2:56 PMI've an issue with the front-end service and I can't loginHi @Jose Infazon, can you elaborate on this point? any screenshot or logs or frontend container would be much appreciated.
j
Jose Infazon
02/17/2023, 3:16 PMThanks @Prashant Shahi. I've restart the ec2 instances and in the frontend I see this:
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "POST /api/v1/login HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/login>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/services>"
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "POST /api/v1/login HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/login>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/services>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "POST /api/v1/login HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/services>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "GET /api/v1/featureFlags HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/featureFlags>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 2023/02/17 15:14:59 [error] 7#7: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "GET /api/v1/configs HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/configs>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 2023/02/17 15:14:59 [error] 7#7: *2 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "GET /api/v1/configs HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/configs>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "GET /api/v1/configs HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/login>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "GET /api/v1/featureFlags HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/featureFlags>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "GET /api/v1/featureFlags HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/login>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"
frontend | 2023/02/17 15:14:59 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "GET /api/v1/version HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/version>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 2023/02/17 15:14:59 [error] 7#7: *1 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "GET /api/v1/version HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/version>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "GET /api/v1/version HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/login>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"That is the log of the frontend
and it is solved if I restart the container
p
Prashant Shahi
02/17/2023, 3:21 PMThat
Not foundj
Jose Infazon
02/17/2023, 3:22 PMbut all the services are up...actually it solves when I only restart the frontend container...so there is my question if I need to add a "depends-on"
p
Prashant Shahi
02/17/2023, 3:24 PMThat should already be set to alertmanager and query-service.
I suppose this issue is happening because query-service got new internal IP assigned while Nginx in frontend is trying to connect to prior internal IP since, it previously was resolve to it.
To verify that, check the internal IP of query-service and the one from frontend logs:
docker inspect $(docker ps -q ) \
--format='{{ printf "%-50s" .Name}} {{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}'Related to this: https://github.com/docker/compose/issues/2003#issuecomment-171184266