Hi, Retention period seems to have reset to default values for some reason. We have set Retention Period following docs @ https://signoz.io/docs/userguide/retention-period/ We set this months ago when we initially installed Signoz as it is recommended to set retention period early in the lifecycle of the platform. In sqlite signoz.db we can see the values: 54 c2da421ae3ae4c1197e0b7ee05377176 2024-03-28 133806.07271329+00:00 2024-03-28 133806.456254169+00:00 signoz_metrics.samples_v2 31104000 604800 success After 7 days(604800) move to S3 cold storage for 1 year(31104000). We have done this for Metrics, Traces and Logs. Since this time(back in March) we have done a number of upgrades of SigNoz and have just noticed there is a change to our Metrics retention period. It is now showing as 1 month and no cold storage setting. See attached image What would have caused this to have changed and why did it only do it for Metrics and not all(Traces and Logs also)? Also we now have months of metrics data, will there be an issue with us changing the retention now?

What would have caused this to have changed and why did it only do it for Metrics and not all(Traces and Logs also)?

We migrated to a new set of tables for metrics.

Also we now have months of metrics data, will there be an issue with us changing the retention now?

What is you overall volume of samples count and time series count?

thanks @Srikanth Chekuri We migrated to a new set of tables for metrics Why did this impact the TTL retention for metrics and not others(traces,logs) and is this now configured somewhere else in signoz.db or in clickhouse?

As I said earlier, we have migrated to new tables with a default TTL of 30 days. It was an overlooked aspect. It should have been considered to use TTL from the past table instead of the default. You should be able to update without issues. The CPU usage might spike when you trigger but after some time it should go back to normal.

@Srikanth Chekuri ok thanks for reply, just to be clear if we change the settings now in signoz ui the value will be saved to the ttl_status table in signoz.db?

yes

@Srikanth Chekuri so we tried to update ttl and getting the following error: {"level":"ERROR","timestamp":"2024-06-20T123346.059Z","caller":"clickhouseReader/reader.go:2432","msg":"error while setting ttl.","error":"code: 47, message: There was an error on [clickhouse9000] Code: 47. DB:Exception *Missing columns: 'timestamp_ms*' while processing query: 'toDateTime(toUInt32(timestamp_ms / 1000), 'UTC') + toIntervalSecond(31104000)', required columns: 'timestamp_ms' 'timestamp_ms'. (UNKNOWN_IDENTIFIER) (version 24.1.2.5 (official build))","stacktrace":"go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).SetTTL.func2\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:2432"}

What version of SigNoz are you using?

v0.46.0

Can you upgrade to 0.47 or 0.48 and try again?

@Srikanth Chekuri we will try this, it will be Monday before we can do it