SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.

SigNoz Community

Hi team. I’m wondering if there is a way to speed up the log search speed when we have a large volume of logs to search from. It seems it trying to find all logs at 1 time rather than find the first page of matched logs and return earlier.

it seems have a default limit 100 for every log query. Is it possible to change to streaming the result so we can see the result asap?. Like if our target log result is 10 lines among 100M lines of logs. currently we have to wait for it to finish scan all the logs instead of quick response.

There is an improved version of logsv2 in upcoming release that does something similar to what you mentioned in original post. It will also be performant when you include resources attributes such as service.name, pod.name etc..

&gt; Like if our target log result is 10 lines among 100M lines of logs. currently we have to wait for it to finish scan all the logs instead of quick response
No, the ClickHouse query doesn't do that when there is a limit. I would say

<@U01NT95GV8B> is there a rough date about when the log change will release so we can adopt it in time.

probably next release or next-to-next release