Yesterday the Signoz VM CPU just spiked to 100% on all 4 cpu cores given to it. Unable to access it ...
o
Yesterday the Signoz VM CPU just spiked to 100% on all 4 cpu cores given to it. Unable to access it through ssh or Proxmox GUI terminal. Memory and disk activity stable the whole time. A soft reboot from Proxmox didn't take so I had to forcefully shut VM off and on again. Even after reboot CPU keept running at full load on all cores for some time, after it settled down. Maybe 5-10 min after reboot. Not sure how to debug it to find the cause. Any suggestion welcome. Has been noticed/reported before?
Seems it was left in a broken state after reboot. Couldn't login any more. Now attempting a re-install
should signoz/migrate keep restarting over and over?
image.png
After re-install I can at least access web gui again.
Updated to 0.54, same state of containers after with migrate restarting and the rest being up.
Might have been my fault for migrate container. I think I noticed a while ago that some containers didn't start automatically. So I ran:
Copy code
docker update --restart unless-stopped $(docker ps --all -q)
I guess this could have caused above seen issue. Whats the default state of containers so I can reset to defaults, as a re-install doesn't seem to change this?
s
Share the error logs of the migrator
o
Like I mentioned, reason for it restarting was most likely on my end due to me changing restart mode to unless-stopped for all containers. A re-install most likely didn't change this as created date say 7 months ago for this contianer. I'm more concerned about why it went to 100 % cpu and stayed there. Though I guess I wont be able to figure that out now. If it happens again I will let you know. Can disregard this one for now.