hi all, just found out about this tool from hacker...
# general
hi all, just found out about this tool from hacker news super excited to start integrating it into our dev and prod environments!! we just moved away from datadog as it was too cost prohibitive for us. Any tips on scaling this out properly with services getting millions of requests an hour would be greatly appreciated!
hey @Vivek Bhakta Welcome 🙂
Few good practices for running in scale: 1. If you are on k8s, start with k8s helm charts - easier to scale with them 2. Create a separate cluster for running SIgNoz, in this way the impact of something going wrong in your app layer is isolated 3. start slow, start with few services first see how much data it is ingesting and based on that set up the disk you want to allocate 4. Set retention periods for metrics, traces and logs - and make sure the storage you allocate is able to handle that
thanks for the tips! we currently run our apis on fargate so we might have to run this on it’s own container