Hi guys, I'm using Vector.dev to forward logs from my app (hosted on Fly.io) to self-hosted Signoz in some VPS. My

vector.toml

is very simple:

[sinks.signoz]
type = "http"
uri = "${PROD_URL:-<http://localhost:8082>}"
inputs = ["hotel"]
encoding.codec = "json"

I followed https://signoz.io/docs/userguide/send-logs-http/ to the dot, except I'm sending the JSON one by one because I don't know how to send "batch" (issue here: https://github.com/vectordotdev/vector/discussions/20425). My problem: Signoz's HTTP endpoint returns 400 if the log comes faster than once a second (I'm exaggerating a bit), which renders it useless. Where can I configure Signoz to allow higher throughput? Error log in Vector (to show you the 400 error):

2024-05-04T14:20:29.727353Z ERROR sink{component_kind="sink" component_id=signoz component_type=http}:request{request_id=1}: vector::sinks::util::retries: Not retriable; dropping the request. reason="Http status: 400 Bad Request" internal_log_rate_limit=true

I know my Vector setup works because it works when I change the sink to e.g., Longtail. Also, if I send request one by one using Postman, Signoz works.

My Clickhouse is configured to put data onto a S3 compatible service (Hetzner Object Storage, I think it's Ceph). I've made the "mistake" of restarting my clickhouse instances. They fail to come up again.@Srikanth Chekuri

This message was deleted.

Hey everyone! We have signoz self hosted set up and running. The dev ops expert set it up for the logs and I wanted to look into the integration for traces. Where it starts and ends is me not being able to find where to create the access token / ingestion key. I can confirm that I am an admin and I do not see a tab under settings. Any ideas?

Just Looking from some support with a local signoz allocation. When trying to perform a query I always get the following error: SELECT DISTINCT JSONExtractString(labels, 'host_name') FROM signoz_metrics.time_series_v2 Received exception from server (version 24.1.2): Code: 173. DB:Exception Received from 127.0.0.1:9000. DB:Exception Couldn't allocate 333 bytes when parsing JSON: while executing 'FUNCTION JSONExtractString(labels :: 0, 'host_name' :: 1) -> JSONExtractString(labels, 'host_name') String : 2'. (CANNOT_ALLOCATE_MEMORY) Any idea how to solve it? Thanks in advance

This message was deleted.

Hi team, I am experience missing traces in signoz. I was using tempo and migrating to signoz. The traces that I was able to find in tempo was not available in signoz. I use opentelemetry collector to send trace to both tempo and signoz. At the time the trace is generated, my opentelemetry collector

otelcol_exporter_send_failed_spans

was not increasing, and on signoz side, the

otelcol_receiver_refused_spans

was 0,

otelcol_receiver_refused_spans

was 0. Is there a way to trouble shoot why the span is missing?

This message was deleted.

This message was deleted.

This message was deleted.