I've setup signoz on an EC2 instance - t2.medium, 40-50 GB EBS. My issue stems from the fact that the system enters into a frozen state and I'm required to reboot the instance in order to ssh and restart Signoz. Signoz is the only "heavyweight" process on that instance. Any body else encounter this issue?
After restarting, I'm also seeing that docker has used all 50GB on the device, how much space does signoz need?
06/28/2023, 2:22 PM
I encountered similar issues when running it for the first time. However, I found a helpful article that provides information on determining the required memory for it to run. I recommend referring to that article for more details.
06/28/2023, 2:28 PM
depends on your workloads and how many traces/metrics you send to it.
The minimum recommended specs for a clickhouse node is 16GB of RAM and 8CPUs I find.
For PROD I run a m6i.2xlarge with a 1.8TB GP3 EBS mount.
I run all of the SigNoz services in ECS to not collocate where Clickhouse is running (the one which needs the most resources)
06/28/2023, 2:32 PM
thanks @Yash RV and @Alexei Zenin! Your suggestions and links seem to refer to Kubernetes deployment. I'm deploying via a standalone docker instance and not leveraging Kubernetes at this time. I've just now turned off the default app and only keeping logs on the system for 1 day - I rely on ELK for logging so no use for SigNoz logs. I'll monitor and check if that helps