Slackbot
12/23/2022, 8:14 AMPranay
moronmon
12/23/2022, 8:20 AMPrashant Shahi
12/23/2022, 8:28 AM1. The traces retention period process is stuck while I change the setting, is there any way to force stop and reset the retention period?Time to enabling TTL depends on amount of data due to limitations of clickhouseDB. How long has been since the TTL operation stuck? If query-service container was not "restarted", TTL operation should either success/fail. If you query-service container was "restarted" after initiation of TTL operation, your checks might not be in-sync with the actual TTL being set in ClickHouse.
Prashant Shahi
12/23/2022, 8:30 AM2. I tried to update Signoz from 0.11.4 to 0.12.0 running using docker-compose.yaml file, but I got an error on otel-collector.It's likely that you have not
git checkout
to v0.12.0
release tag. There is additional clickhouse-cluster.xml
file which includes ClickHouse cluster information.
https://github.com/SigNoz/signoz/tree/develop/deploy/docker/clickhouse-setupmoronmon
12/23/2022, 8:51 AMmoronmon
12/23/2022, 8:53 AMPrashant Shahi
12/23/2022, 9:16 AMdocker-compose.yaml
like Zookeeper dependency, clickhouse-cluster.xml
volume mounting, etc.
Make sure the YAML is in-sync if not identical from the one in develop
.
https://github.com/SigNoz/signoz/blob/develop/deploy/docker/clickhouse-setup/docker-compose.yamlPrashant Shahi
12/23/2022, 9:18 AMIt's been a while, and I don't remember exactly how long, but I just notice the storage usage of ClickhouseDB getting bigger, and when I open the settings page, I saw the changes of the retention period still ongoing. I already restart all the container, but the process not stopped.Can you share the screenshot? If it are sure that it is stuck. You might want to delete the TTL operation checks from SQLite and apply it again. cc @Vishal Sharma @nitya-signoz
moronmon
12/23/2022, 9:41 AMmoronmon
12/23/2022, 9:42 AMPrashant Shahi
12/23/2022, 9:50 AMmoronmon
12/23/2022, 9:51 AMmoronmon
12/23/2022, 9:51 AMVishal Sharma
12/23/2022, 9:55 AMmoronmon
12/23/2022, 9:56 AMVishal Sharma
12/23/2022, 10:00 AMdocker exec -it query-service sh
Run the following:
# install sqlite
apk update
apk add sqlite
# open sqlite with signoz.db
sqlite3 /var/lib/signoz/signoz.db
# (sqlite shell) check existing ttl status
select * from ttl_status;
# (sqlite shell) delete all ttl status
DELETE from ttl_status where 1=1
# (sqlite shell) verify ttl status deletion
select * from ttl_status;
moronmon
12/23/2022, 10:10 AMmoronmon
12/23/2022, 10:10 AMPranay
K Kumar
03/11/2024, 8:46 AMK Kumar
03/11/2024, 8:46 AMK Kumar
03/11/2024, 10:46 AM