Hi team! @nitya-signoz We recently implemented the SigNoz APM service (Chart signoz-0.43.0-app version 0.47.0) in a production Java application running with 2 replicas on EKS. This service previously used Datadog APM. The idea was to verify that SigNoz metrics align with the expected values. The application experiences reduced traffic and latency during certain nighttime hours, but SigNoz reported very high latency during this period. The

click-house-cluster

pod was configured with a limit of 4 CPU and 5 GB of memory, and the

otel-collector

pod has a limit of 500 m CPU and 4 GB of memory. These pods generally operate well within these resources, but the

otel-collector

logs the following warning every minute. I am unsure if this is normal or if additional configuration adjustments are needed for SigNoz or Should any configuration be done in the otel-collector metrics?:

2024-07-12T17:52:47.600Z warn clickhousemetricsexporter/exporter.go:272 Dropped cumulative histogram metric {"kind": "exporter", "data_type": "metrics", "name": "clickhousemetricswrite", "name": "signoz_latency"}
2024-07-12T17:52:51.805Z warn clickhousemetricsexporter/exporter.go:279 Dropped exponential histogram metric with no data points {"kind": "exporter", "data_type": "metrics", "name": "clickhousemetricswrite", "name": "signoz_latency"}

``` ```

You ignore those logs as they are just warnings. cc @Srikanth Chekuri

These are non-harmful warnings. You can create an issue here https://github.com/SigNoz/signoz to get rid of those warnings.