https://signoz.io logo
Join Slack
Powered by
Hi everyone, yesterday I changed the log retention...
# support
k
Hi everyone, yesterday I changed the log retention period, but it gave me an error. I also ran a few queries in ClickHouse (not very complex). After that, the CPU usage became very high, and no logs were coming in. The connection to 
<http://xxx.xx.xx.xx:4318/v1/traces>
timed out. After restarting Docker, also kill some processes started coming in again. How can I identify the root cause, and how can I resolve this issue without restarting Docker? Also, how can I know if this issue has occurred before? Please support me. Thanks.
Copy code
m: Poco::Exception. Code: 1000, e.code() = 32, I/O error: Broken pipe, Stack trace (when copying this message, always include the lines below):
2025.02.12 02:44:48.816925 [ ^[[38;2;25;155;253m764^[[0m ] {^[[38;2;38;190;41m^[[0m} <^[[1mInformation^[[0m> ^[[38;2;248;108;0mEmbeddedDictionaries^[[0m: Loading dictionaries.
2025.02.12 02:44:48.817025 [ ^[[38;2;25;155;253m764^[[0m ] {^[[38;2;38;190;41m^[[0m} <^[[1mInformation^[[0m> ^[[38;2;248;108;0mEmbeddedDictionaries^[[0m: Loaded dictionaries.
2025.02.12 02:44:56.349005 [ ^[[38;2;198;84;166m2089630^[[0m ] {^[[38;2;0;202;216m725ab2a3-cd08-4f6d-a293-4bb6065cdb21^[[0m} <^[[1mInformation^[[0m> ^[[38;2;70;122;255mTCPHandler^[[0m: Change cancellation status from NOT_CANCELLED to FULLY_CANCELLED. Log message: Query was cancelled.
2025.02.12 02:44:56.354751 [ ^[[38;2;198;84;166m2089630^[[0m ] {^[[38;2;0;202;216m725ab2a3-cd08-4f6d-a293-4bb6065cdb21^[[0m} <^[[1;31mError^[[0m> ^[[38;2;57;146;214mexecuteQuery^[[0m: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket (172.18.0.5:9000 -> 172.18.0.6:52970). (NETWORK_ERROR) (version 24.1.2.5 (official build)) (from 172.18.0.6:52970) (comment: {"alertID":"","client":"browser","dashboardID":"","path":"/logs/logs-explorer","servicesTab":"","source":"logs-explorer","viewName":""}) (in query: SELECT toStartOfInterval(fromUnixTimestamp64Nano(timestamp), INTERVAL 8640 SECOND) AS ts, toFloat64(count(*)) as value from signoz_logs.distributed_logs where (timestamp >= 1736736312000000000 AND timestamp <= 1739328312000000000) group by ts order by value DESC), Stack trace (when copying this message, always include the lines below):
2025.02.12 02:44:56.356162 [ ^[[38;2;198;84;166m2089630^[[0m ] {^[[38;2;0;202;216m725ab2a3-cd08-4f6d-a293-4bb6065cdb21^[[0m} <^[[1;31mError^[[0m> ^[[38;2;70;122;255mTCPHandler^[[0m: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket (172.18.0.5:9000 -> 172.18.0.6:52970). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below):
2025.02.12 02:44:56.356524 [ ^[[38;2;198;84;166m2089630^[[0m ] {^[[38;2;0;202;216m725ab2a3-cd08-4f6d-a293-4bb6065cdb21^[[0m} <^[[0;31mWarning^[[0m> ^[[38;2;70;122;255mTCPHandler^[[0m: Client has gone away.
2025.02.12 02:45:44.743852 [ ^[[38;2;90;164;41m1861376^[[0m ] {^[[38;2;38;190;41m^[[0m} <^[[1;31mError^[[0m> ^[[38;2;0;172;236mServerErrorHandler^[[0m: Poco::Exception. Code: 1000, e.code() = 32, I/O error: Broken pipe, Stack trace (when copying this message, always include the lines below):
s
What version of SigNoz are you on?
k
v0.54.0
s
Please upgrade to the latest version. The way TTL updates has been changed. It wouldn't throw error or use high resources in latest version.
11 Views
Open in Slack
PreviousNext
Powered by