Hey folks! I’ve been looking at Clickhouse for observability data and ran into Signoz soon enough! Thanks for creating this neat project 🙂 I’m particularly looking at Logs for now and was wondering about two things that I plan to play around/benchmark “soon”. : • If I understand correctly, currently there’s a materialized column that’s created when we want to query on a particular label (attribute key) . Is it possible to use projections in this scenario? • Has it been considered using inverted indices on the Body? https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/clickhouseexporter/exporter_logs.go#L143

The user is given an option to materialize a field for better query perf or usage in dashboards you can read more about it here https://signoz.io/docs/userguide/logs_fields/ While we are keeping an eye on inverter indices, but as of now we are prioritizing on other things. You can check our perf benchmark here https://signoz.io/blog/logs-performance-benchmark/

Thanks for responding!

materialize a field for better query perf or usage in dashboards

I’d like to give my users a read-only credential* to the the DB server as I don’t want them to start relying on creating tables/columns I don’t want to maintain. It’s why I was considering projections as Clickhouse could then optimise the query internally.

We have our own distro of otel collector you can check here

Amazing, I think the Makefile in there should let me build and play around. I’m also looking into opamp, are there any high level deployment docs around that? *- We want to store other data in Clickhouse and provide a single interface for querying/building on top of it, so we might end up in a situation where we can’t use Signoz UI or the query service. Not sure how it’ll play out but that’s kinda the end goal. Proobably, I might have to consider using query-service and extending it in some way…

We want to store other data in Clickhouse and provide a single interface for querying/building on top of it

@Harsh Thakur you can create a dashboard by writing clickhouse sql query. So, you should be able to plot chart on non-observability data too. What are your use cases?

Haven’t decided on the UI yet but generally besides our core dashboarding (might even be Signoz UI itself), we want to provide direct Clickhouse access(read-only) so users can feel free to chose their dashboarding component- Metabase, superset, etc.

the whole interactiveness of the observability suite would be lost.

I do want this + correlation with other data too… and that’s where I wonder if Signoz UI would be the right component.

As in from Signoz UI?

Yes

Um, so the Auth layer is at query service and UI tho, right?

correlation with other data too

We have been looking for usecases to build this. Mind to discuss more over a call? https://calendly.com/ankit-nayan/observability-discussions-with-signoz

Thank you, that would be great! I’ll chat with my team, check their availability and schedule a call.

looking forward for the call 🙂