Hello Team, I’m working on a project that requires ingesting 30 days of historical data from our finance team into a ClickHouse database. The challenge is that this data must be visualized on the dashboard as if it were ingested on an hourly basis. If I follow the standard hourly ingestion, the process will take 30 days to complete. However, our requirement is to ingest the entire dataset immediately while preserving the original hourly timestamps to ensure accurate representation on the dashboard. Could you please guide me on the best way to: 1. Bulk ingest this historical data into ClickHouse. 2. Maintain the timestamps so that the data appears as if it were ingested hourly on the dashboard. Your assistance in resolving this would mean a lot to me.

You need to send data to SigNoz with the original timestamps, that is not possible with using the SDKs, you need to use the protobufs libraries to generate messages and send them to the collector.

@Srikanth Chekuri Could you please provide more details or resources on how to implement this approach effectively? Any examples or documentation would be greatly appreciated to help me get started.

@Farzana Shaikh please stop pinning message to channel

@Srikanth Chekuri I sincerely apologize for causing any inconvenience to you or the channel from my side. It was not my intention, but I pinned the message because it was important for my work. I truly appreciate your guidance, help, and response, and I assure you that I will not pin messages to the channel going forward.

The most relevant example I can give is you look into how these libraries prepare and send data https://github.com/open-telemetry/opentelemetry-python/tree/main/exporter/opentelemetry-exporter-otlp