Hello, we have been using self hosted signoz (running on EKS) in our dev env for our web app that uses aws lamda. We recently added signoz for prod env and see lot of HTTP connection failures trying to reach signoz endpoint. Tried increasing otel-collector pod replicas but does not seem to help. The CPU of node where pods are are scheduled is high at 95%. What could be other options to scale signoz ?
r
Rajat Dabade
03/20/2024, 9:13 AM
@Prashant Shahi Can you please take a look. Thanks.
p
Prashant Shahi
03/20/2024, 4:23 PM
You can scale up the ClickHouse shards.
Prashant Shahi
03/20/2024, 4:24 PM
But before that, you would want to monitor which pods are taking up more resource.
SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.