My ClickHouse pod is taking a way to much storage, like in 3-4 days, it has taken about 25 tb disk storage, so is there is any way to reduce this, or any insight to reduce this much storage consumption or we can compress the data, and this is happening when only traces is allowed, currently i have disabled logs and metrics till now, and if in future if i allow that also, it will take much storage, so anyone can suggest that how we can reduce a storage consumption of clickhouse? @Chitransh Gupta @Prashant Shahi, your insight will be very helpful.

Clickhouse has very efficient compression underneath so 25TB is insane. Have you turned off the hotrod and load-hotrod services? They generate tons of dummy data, and if running in a hyper-scaled kubernetes environment, it can fill storage faster. Please, share your service map/architecture here if possible.

Thanks for your response @Ravi Semwal, yes i have turned off hotrod and load-hotrod services, and i am running signoz in hyper-scaled kubernetes envirnment, due to security constrain i can not share my architecture, but here is something i have tried to apply, i have used a gcs bucket to store a data after particular amount of disk is full i have used this config in values.yaml file ------------------------------------ clickhouse: coldStorage: enabled: true # Set free space size on default disk in bytes defaultKeepFreeSpaceBytes: "10485760" # 10MiB type: gcs endpoint: https://storage.googleapis.com/<bucket-name>/data/ accessKey: <access_key_id> secretAccess: <secret_access_key> ---------------------------------------- but after apply this when i set a retention period from an UI, i am getting this issue, Error:- "Your last call to change retention period to 7 days and S3 to 2 days failed. Please try again." In values.yaml file, i have set config of an gcs, but in UI its still showing me a s3.. @Srikanth Chekuri can you look into this.

First thing would be investigating what is exactly taking a lot of storage. After we pinpoint the nature of such data, we could either filter it out or disable it during the collection/db write.