Hello, i seem to be having issues with sending queue is full `{"level":"error","ts":1727180784.2226...
s

Samuel Olowoyeye

7 months ago
Hello, i seem to be having issues with sending queue is full
{"level":"error","ts":1727180784.2226639,"caller":"exporterhelper/common.go:296","msg":"Exporting failed. Rejecting data.","kind":"exporter","data_type":"traces","name":"clickhousetraces","error":"sending queue is full","rejected_items":30,"stacktrace":"<http://go.opentelemetry.io/collector/exporter/exporterhelper.(*baseExporter).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*baseExporter).send>\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.102.0/exporterhelper/common.go:296\<http://ngo.opentelemetry.io/collector/exporter/exporterhelper.NewTracesRequestExporter.func1|ngo.opentelemetry.io/collector/exporter/exporterhelper.NewTracesRequestExporter.func1>\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/exporter@v0.102.0/exporterhelper/traces.go:134\<http://ngo.opentelemetry.io/collector/consumer.ConsumeTracesFunc.ConsumeTraces|ngo.opentelemetry.io/collector/consumer.ConsumeTracesFunc.ConsumeTraces>\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/consumer@v0.102.1/traces.go:25\<http://ngo.opentelemetry.io/collector/processor/batchprocessor.(*batchTraces).export|ngo.opentelemetry.io/collector/processor/batchprocessor.(*batchTraces).export>\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.102.0/batch_processor.go:414\<http://ngo.opentelemetry.io/collector/processor/batchprocessor.(*shard).sendItems|ngo.opentelemetry.io/collector/processor/batchprocessor.(*shard).sendItems>\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.102.0/batch_processor.go:261\<http://ngo.opentelemetry.io/collector/processor/batchprocessor.(*shard).startLoop|ngo.opentelemetry.io/collector/processor/batchprocessor.(*shard).startLoop>\n\t/home/runner/go/pkg/mod/go.opentelemetry.io/collector/processor/batchprocessor@v0.102.0/batch_processor.go:223"}
However, when i checked clickhouse storage i got
Filesystem                Size      Used Available Use% Mounted on
/dev/sde                884.8G    265.1G    574.7G  32% /var/lib/clickhouse
chi-my-release-clickhouse-cluster-0-0-0:/$
What could be the issue here
Hello, I'm facing an issue when deploying signoz in deattached mode. that is running it as `docker c...
j

JAIME SEBASTIAN BURBANO VILLAVICENCIO

12 months ago
Hello, I'm facing an issue when deploying signoz in deattached mode. that is running it as
docker compose -f docker-compose.yaml up
Note: I have almost not changed the compose file at all, just commented the hotrop example. Every some seconds, the following error appears in clickhouse
signoz-clickhouse      | 2024.05.13 07:45:46.188830 [ 47 ] {} <Error> ServerErrorHandler: Poco::Exception. Code: 1000, e.code() = 32, I/O error: Broken pipe, Stack trace (when copying this message, always include the lines below):
signoz-clickhouse      | 
signoz-clickhouse      | 0. Poco::Net::SocketImpl::error(int, String const&) @ 0x00000000153a1b5f in /usr/bin/clickhouse
signoz-clickhouse      | 1. Poco::Net::SocketImpl::sendBytes(void const*, int, int) @ 0x00000000153a2bbd in /usr/bin/clickhouse
signoz-clickhouse      | 2. Poco::Net::StreamSocketImpl::sendBytes(void const*, int, int) @ 0x00000000153a5296 in /usr/bin/clickhouse
signoz-clickhouse      | 3. Poco::Net::HTTPSession::write(char const*, long) @ 0x00000000153908b3 in /usr/bin/clickhouse
signoz-clickhouse      | 4. Poco::Net::HTTPHeaderIOS::~HTTPHeaderIOS() @ 0x000000001538bbdb in /usr/bin/clickhouse
signoz-clickhouse      | 5. Poco::Net::HTTPHeaderOutputStream::~HTTPHeaderOutputStream() @ 0x000000001538bf1f in /usr/bin/clickhouse
signoz-clickhouse      | 6. DB::HTTPServerResponse::send() @ 0x0000000012942988 in /usr/bin/clickhouse
signoz-clickhouse      | 7. DB::HTTPServerConnection::sendErrorResponse(Poco::Net::HTTPServerSession&, Poco::Net::HTTPResponse::HTTPStatus) @ 0x000000001293ecda in /usr/bin/clickhouse
signoz-clickhouse      | 8. DB::HTTPServerConnection::run() @ 0x000000001293e97b in /usr/bin/clickhouse
signoz-clickhouse      | 9. Poco::Net::TCPServerConnection::start() @ 0x00000000153a5a72 in /usr/bin/clickhouse
signoz-clickhouse      | 10. Poco::Net::TCPServerDispatcher::run() @ 0x00000000153a6871 in /usr/bin/clickhouse
signoz-clickhouse      | 11. Poco::PooledThread::run() @ 0x000000001549f047 in /usr/bin/clickhouse
signoz-clickhouse      | 12. Poco::ThreadImpl::runnableEntry(void*) @ 0x000000001549d67d in /usr/bin/clickhouse
signoz-clickhouse      | 13. ? @ 0x0000721d932f8609
signoz-clickhouse      | 14. ? @ 0x0000721d9321d353
signoz-clickhouse      |  (version 24.1.2.5 (official build))
My problem is that I can get trace information in clickhouse but cannot get any hostmetrics. I have a local collector running in a different machine, which I can assure it is collecting data each 10 seconds (as I can see it in the logger), but this data does not seem to reach clickhouse DB. I have no idea if it is related to the error shown above. Any help or guidance is appreciated.