i m trying to use kafka as a log receiver to collect telemetry events from application and then consume from that topic to export all data to the clickhouse db so i after reading about otel configuration and opentelemetry maven plugin , so i created one dedicated topic for this purpose and configured this in my sample spring boot application and provided kafka broker address and topic name in the application.properties so that opntelemetry can push telemetry events to kafka and also added kafka broker address and topic name in signoz-collector.yaml file so that it can read the data from topic and export to clickhouse but what problem i m constantly facing is that , i m not able to receive any event on topic and also opntelemetry is always trying to connect with localhost:4317 which is deafault grpc endpoint reciever in otlp so not able to ovveride that default configuration so can anyone give any input on the same if anyone tried to use opentelemetry maven plugin with kafka exporter ?

@sarthak gupta Are you trying to send directly from Kafka topic to ClickHouse DB or you are sending from Kafka topic to otel collector to ClickHouse? Also, curious why are you trying ingest to Kafka? What's your use case?

Hi Pranay , actually initially i deployed signoz with it's default grpc endpoint otel receiver but what i observed when i instrumented one of high throuput microservices is when signoz cluster was down due to high cpu utilisation , the otel agent was continuously throwing that grpc error as was not able to make a connection and after 1,2 days i found service unhealthy so it was affecting the source application performance during signoz downtime , so after doing some research on opntelemetry, if found that we can also use kafka as a telemetry event receiver by configuring broker address and topic name in source application so that i can decouple the service directly with signoz and produce data in async mode into kafka which can be read by signoz to export to clickhouse .

If I understand correctly this is your flow you want to achieve otel_events(application) -> Kafka -> otel_collector -> clickhouse is what you are trying to acheive right ?