I ve tried updating the retention period for traces from 2 d SigNoz Community #support

I’ve tried updating the retention period for trace...

Alexei Zenin

12/12/2022, 3:08 PM

I’ve tried updating the retention period for traces from 2 days to 1 month and it has been trying to apply this change since Friday (taking 3 days). anything i can do to fix this? Seems its been retaining data for more than 2 days now but still says its applying

Pranay

12/12/2022, 3:32 PM

@Vishal Sharma any insights here?

Alexei Zenin

12/12/2022, 4:23 PM

Previous discussion about this: https://signoz-community.slack.com/archives/C01HWQ1R0BC/p1668716164380199

Alexei Zenin

12/12/2022, 9:49 PM

I ended up nuking the clickhouse instance to resize it to a larger EC2 instance type and the SigNoz UI seems to have remembered it was still pending on changing retention to 30 days and I can’t seem to set it to 30. Is there some kind of timer where it keeps retrying in the signoz app? Can i somehow wipe or reset this without nuking the entire signoz DB?

Alexei Zenin

12/12/2022, 9:56 PM

I ended up nuking the SigNoz services + filesystem on EFS to start fresh. Hoping this gets looked at as when this actually gets used in PROD would not want to send an invite link to everyone again

Vishal Sharma

12/13/2022, 2:49 AM

@Alexei Zenin TTL is an async operation which is why it’s taking so long. If you want to make the changes faster then you can manually connect to clickhouse and then reduce

merge_with_ttl_timeout

to 10mins, this has performance impact as TTL will be executed frequently. Performance impact of this can be reduced by enabling

ttl_only_drop_parts

. Also, if your table is not too big, you could force it using the

OPTIMIZE TABLE [db.]table FINAL

statement. However, for large tables, it is not recommended.

Vishal Sharma

12/13/2022, 2:57 AM

Command to set

merge_with_ttl_timeout

to 10mins:

Copy code

SET merge_with_ttl_timeout = 600

Command to enable

ttl_only_drop_parts

Copy code

SET ttl_only_drop_parts = 1

If you want to run

OPTIMIZE TABLE [db.]table FINAL

for all the trace tables then run below queries one by one: 1.

OPTIMIZE TABLE signoz_traces.signoz_error_index_v2 FINAL

OPTIMIZE TABLE signoz_traces.usage_explorer FINAL

OPTIMIZE TABLE signoz_traces.dependency_graph_minutes FINAL

OPTIMIZE TABLE signoz_traces.signoz_spans FINAL

OPTIMIZE TABLE signoz_traces.signoz_index_v2 FINAL

OPTIMIZE TABLE signoz_traces.durationSort FINAL

11 Views

Open in Slack

Previous Next