We ran a query(see below) on logs for a 2 month time range 9 June 2024 14:50:22 to Thursday, 8 Augus...
n
We ran a query(see below) on logs for a 2 month time range 9 June 2024 145022 to Thursday, 8 August 2024 145022. Our Logs retention is 12 months and move to s3 after 7 days. Current volume of logs is about 1.3 million a days. SigNoz is running on a m7a.large aws ec2 instance(2 CPU, 8GB memory) We are monitoring this EC2 instance with Node Exporter and when the query ran the CPU and Memory went high(see image), resulting in SigNoz being unresponsive. We are also sending the same logs to Wazuh and ran the same query here and it returned the result in 5 seconds. To resolve this we had to restart the ec2 instance. We would have expected this query to run and maybe result in CPU and memory going up slightly but not reaching 100% and causing SigNoz to be unresponsive. Is our expectation wrong? Query: _{"level":"INFO","timestamp":"2024-08-08T135030.340Z","caller":"utils/time.go:17","msg":"Elapsed time","func_name":"GetTimeSeriesResultV3","duration":8164,"path":"/logs/logs-explorer","dashboardID":"","alertID":"","source":"logs-explorer","client":"browser","viewName":"","servicesTab":"","query":"SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 17280 SECOND) AS ts, toFloat64(count(*)) as value from signoz_logs.distributed_logs where (timestamp >= 1717941022000000000 AND timestamp <= 1723125022000000000) group by ts order by value DESC"}_
s
We are also sending the same logs to Wazuh and ran the same query here and it returned the result in 5 seconds.
I am not familiar with this. Is it also an open-source setup with the same resource provisioning?