Hi Everyone - I've deployed signoz on an AWS EC2 i...
# generalj
Jose Infazon
02/17/2023, 11:50 AMHi Everyone - I've deployed signoz on an AWS EC2 instance. This instance is scheduled to start and shutdown at certain hours. When it starts, all docker signoz containers start (I've set restart: unless stopped" in the docker compose. But I've an issue with the front-end service and I can't login, I always need to restart front-end service. Is there any dependency I need to add?
v
Vishal Sharma
02/17/2023, 1:26 PM@Prashant Shahi Can you please help him here?
p
Prashant Shahi
02/17/2023, 2:56 PMI've an issue with the front-end service and I can't loginHi @Jose Infazon, can you elaborate on this point? any screenshot or logs or frontend container would be much appreciated.
j
Jose Infazon
02/17/2023, 3:16 PMThanks @Prashant Shahi. I've restart the ec2 instances and in the frontend I see this:
Jose Infazon
02/17/2023, 3:17 PM Copy code
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "POST /api/v1/login HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/login>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/services>"
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "POST /api/v1/login HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/login>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/services>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "POST /api/v1/login HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/services>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "GET /api/v1/featureFlags HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/featureFlags>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 2023/02/17 15:14:59 [error] 7#7: *2 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "GET /api/v1/configs HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/configs>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 2023/02/17 15:14:59 [error] 7#7: *2 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "GET /api/v1/configs HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/configs>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "GET /api/v1/configs HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/login>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"
frontend | 2023/02/17 15:14:59 [error] 6#6: *3 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "GET /api/v1/featureFlags HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/featureFlags>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "GET /api/v1/featureFlags HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/login>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"
frontend | 2023/02/17 15:14:59 [error] 7#7: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 52.8.17.212, server: _, request: "GET /api/v1/version HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/version>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 2023/02/17 15:14:59 [error] 7#7: *1 open() "/usr/share/nginx/html/50x.html" failed (2: No such file or directory), client: 52.8.17.212, server: _, request: "GET /api/v1/version HTTP/1.1", upstream: "<http://172.18.0.5:8080/api/v1/version>", host: "<http://signoz-dev.kalto.la:3301|signoz-dev.kalto.la:3301>", referrer: "<http://signoz-dev.kalto.la:3301/login>"
frontend | 52.8.17.212 - - [17/Feb/2023:15:14:59 +0000] "GET /api/v1/version HTTP/1.1" 404 185 "<http://signoz-dev.kalto.la:3301/login>" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36" "-"Jose Infazon
02/17/2023, 3:17 PMThat is the log of the frontend
Jose Infazon
02/17/2023, 3:17 PMand it is solved if I restart the container
p
Prashant Shahi
02/17/2023, 3:21 PMThat
Not foundj
Jose Infazon
02/17/2023, 3:22 PMbut all the services are up...actually it solves when I only restart the frontend container...so there is my question if I need to add a "depends-on"
p
Prashant Shahi
02/17/2023, 3:24 PMThat should already be set to alertmanager and query-service.
Prashant Shahi
02/17/2023, 3:29 PMI suppose this issue is happening because query-service got new internal IP assigned while Nginx in frontend is trying to connect to prior internal IP since, it previously was resolve to it.
Prashant Shahi
02/17/2023, 3:29 PMTo verify that, check the internal IP of query-service and the one from frontend logs:
Copy code
docker inspect $(docker ps -q ) \
--format='{{ printf "%-50s" .Name}} {{range .NetworkSettings.Networks}}{{.IPAddress}} {{end}}'Prashant Shahi
02/17/2023, 3:49 PMRelated to this: https://github.com/docker/compose/issues/2003#issuecomment-171184266
10 Views