https://signoz.io logo
Join Slack
Powered by
Hey everyone, Our team is not able to run this que...
# support
a
Hey everyone, Our team is not able to run this query or rather visualise it, the list view works fine but aggregate operator like P95 or Count don't work at all neither do the time series or table view.
n
can you capture the api response of the query range api and share it ?
a
Copy code
<html>
<head><title>504 Gateway Time-out</title></head>
<body>
<center><h1>504 Gateway Time-out</h1></center>
<hr><center>nginx/1.18.0 (Ubuntu)</center>
</body>
</html>
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
<!-- a padding to disable MSIE and Chrome friendly error page -->
The api fails every time
n
what is the timerange you have selected ? also can you share logs of query service when you trigger this api ?
a
time range - 
Last 4 Days
time range - 
Last 4 Days
recent logs after calling query_range api
Copy code
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:55:43.465Z","caller":"app/server.go:395","msg":"/api/v3/query_range","timeTaken":0,"path":"/api/v3/query_range"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:55:43.706Z","caller":"app/server.go:395","msg":"/api/v1/login","timeTaken":0,"path":"/api/v1/login"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:55:43.981Z","caller":"app/server.go:395","msg":"/api/v1/user/{id}","timeTaken":0,"path":"/api/v1/user/{id}"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:55:44.092Z","caller":"app/server.go:395","msg":"/api/v1/version","timeTaken":0,"path":"/api/v1/version"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:55:44.215Z","caller":"app/server.go:395","msg":"/api/v1/featureFlags","timeTaken":0,"path":"/api/v1/featureFlags"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:55:44.457Z","caller":"app/server.go:395","msg":"/api/v2/licenses","timeTaken":0,"path":"/api/v2/licenses"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:55:44.542Z","caller":"app/server.go:395","msg":"/api/v1/featureFlags","timeTaken":0,"path":"/api/v1/featureFlags"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:56:13.070Z","caller":"app/server.go:395","msg":"/api/v1/health","timeTaken":0,"path":"/api/v1/health"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:56:43.184Z","caller":"app/server.go:395","msg":"/api/v1/health","timeTaken":0,"path":"/api/v1/health"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"ERROR","timestamp":"2024-11-05T06:56:44.323Z","caller":"clickhouseReader/reader.go:4454","msg":"error while reading result","error":"context canceled","stacktrace":"<http://go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError|go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError>\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4454\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.readRowsForTimeSeriesResult\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4308\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).GetTimeSeriesResultV3\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4391\ngo.signoz.io/signoz/pkg/query-service/app/querier.(*querier).execClickHouseQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/querier.go:106\ngo.signoz.io/signoz/pkg/query-service/app/querier.(*querier).runBuilderQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/helper.go:202"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:56:44.323Z","caller":"utils/time.go:17","msg":"Elapsed time","func_name":"GetTimeSeriesResultV3","duration":59973,"path":"/traces-explorer","dashboardID":"","alertID":"","source":"traces-explorer","client":"browser","viewName":"\"\"","servicesTab":"","query":"SELECT toStartOfInterval(timestamp, INTERVAL 1140 SECOND) AS ts, toFloat64(count()) as value from signoz_traces.distributed_signoz_index_v2 where (timestamp >= '1730443620000000000' AND timestamp <= '1730789040000000000') AND serviceName IN ['phot-ai-production'] AND stringTagMap['http.route'] ILIKE '%/studio%' group by ts order by value DESC"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T06:56:44.323Z","caller":"app/server.go:395","msg":"/api/v3/query_range","timeTaken":60000,"path":"/api/v3/query_range"}
n
The query is timing out, I guess it works when you select a smaller timerange ? If yes then you will have to give more resources to clickhouse as of now. We are doing some improvements for search in traces but it will take some time to be released. You can track this PR for that https://github.com/SigNoz/signoz/issues/5713
a
It's still timing out when I select 
Last 1 Day
Do we need to lower the range even further or increase the resource?
Copy code
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"ERROR","timestamp":"2024-11-05T07:02:53.193Z","caller":"clickhouseReader/reader.go:4454","msg":"error while reading result","error":"context deadline exceeded","stacktrace":"<http://go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError|go.signoz.io/signoz/pkg/query-service/app/clickhouseReader.getPersonalisedError>\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4454\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.readRowsForTimeSeriesResult\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4308\ngo.signoz.io/signoz/pkg/query-service/app/clickhouseReader.(*ClickHouseReader).GetTimeSeriesResultV3\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:4391\ngo.signoz.io/signoz/pkg/query-service/app/querier.(*querier).execClickHouseQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/querier.go:106\ngo.signoz.io/signoz/pkg/query-service/app/querier.(*querier).runBuilderQuery\n\t/home/runner/work/signoz/signoz/pkg/query-service/app/querier/helper.go:202"}
signoz_query-service.1.i4gkv1u2xc3u@ip-172-31-29-251    | {"level":"INFO","timestamp":"2024-11-05T07:02:53.193Z","caller":"utils/time.go:17","msg":"Elapsed time","func_name":"GetTimeSeriesResultV3","duration":60042,"client":"browser","viewName":"\"\"","servicesTab":"","query":"SELECT toStartOfInterval(timestamp, INTERVAL 240 SECOND) AS ts, toFloat64(count()) as value from signoz_traces.distributed_signoz_index_v2 where (timestamp >= '1730703600000000000' AND timestamp <= '1730790000000000000') AND serviceName IN ['phot-ai-production'] AND stringTagMap['http.route'] ILIKE '%/studio%' group by ts order by value DESC","path":"/traces-explorer","dashboardID":"","alertID":"","source":"traces-explorer"}
n
Try selecting a 15/30 mins range to get the idea ? then increase the range, also how much spans are you sending ? you can get that in the timeseries page without selecting any filters.
8 Views
Open in Slack
PreviousNext
Powered by