Hi. We upgraded signoz from version 0.63.0 to vers...
# support
w
Hi. We upgraded signoz from version 0.63.0 to version 0.64.0 (I know it's an old version but we are limited to upgrading versions by signoz users) and we are getting errors in dashboards: API responded with 500 - Something went wrong status: error in prom queries. [screen] But after 1 minute hitting refresh on the same dashboard shows results. The same dashboard didn't have issues in 0.63.0. How can we debug this? I tried searching logs in clickhouse, query-service and frontend but I don't see anything related. Clickhouse shows this kind of error, but I don't know if it's related (we are using 2shards and 2replica setup in k8s, IPsbelow are IP of two clickhouse-cluster pods):
Copy code
{
  "date_time": "1743999075.303117",
  "thread_name": "",
  "thread_id": "782",
  "level": "Information",
  "query_id": "d278e7f0-9a49-4ba8-ae45-9d303ed0174b",
  "logger_name": "HedgedConnectionsFactory",
  "message": "Connection failed at try №1, reason: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket (10.42.14.131:47880 -> 10.42.16.168:9000). (NETWORK_ERROR) (version 24.1.2.5 (official build))",
  "source_file": "src/Client/HedgedConnectionsFactory.cpp; HedgedConnectionsFactory::State DB::HedgedConnectionsFactory::processFinishedConnection(int, TryResult, Connection *&)",
  "source_line": "327"
}
Other dashboards seem to work fine and not throw errors. If I need to send more info, please tell.