Nick Burrett03/29/2023, 1:32 PM
Srikanth Chekuri03/29/2023, 2:38 PM
CPU utilisation seem very high for a database that i would consider to be receiving little in the way of updates.Which updates are you referring? We don’t push any
commands in SigNoz and where do you see the
wise CPU utilization? You can use the Unix tool ``sudo perf top`` to get the operation which are taking time. In the past we have noticed this happens when the
which is responsible for merge the many temporary files to single data block responsible for this.
Nick Burrett03/29/2023, 2:48 PM
statement, which is effectively an update to a database.
INSERT INTO signoz_logs
Srikanth Chekuri03/29/2023, 2:54 PM
to check which is claiming more CPU.
Nick Burrett03/29/2023, 2:57 PM
SQL statements are entirely different things but the modifiication of a database in any way is an update to a database (the database has been updated with new data). This sounds like an argument over nomenclature. Each node as 4 CPUs. The write write is 3700 logs per minute (~61 per second) The IO utilisation on the server is less than 5%, with write-queue latency at 12ms.
isn't giving useful information due to lack of access to any symbols at the moment
Srikanth Chekuri03/29/2023, 3:01 PM
Nick Burrett03/29/2023, 3:03 PM
Srikanth Chekuri03/29/2023, 3:05 PM
Nick Burrett03/29/2023, 3:06 PM
Srikanth Chekuri03/29/2023, 3:07 PM
Nick Burrett03/29/2023, 3:15 PM
And more than 0.5 seconds to run:
SELECT DISTINCT fingerprint FROM signoz_metrics.distributed_time_series_v2
So it's generally just slow. But that's compounded by because of the full CPU utilisation of the clickhouse-service on the VM.
INSERT INTO signoz_logs.distributed_log
Srikanth Chekuri03/29/2023, 3:25 PM
Nick Burrett03/29/2023, 3:27 PM
Srikanth Chekuri03/29/2023, 3:40 PM
Nick Burrett03/31/2023, 9:36 AM