Hello we are seeing a lot of the following logs in our `signoz-collector` running in K8s. We gave th...
a

Apoorva Gupta

10 months ago
Hello we are seeing a lot of the following logs in our
signoz-collector
running in K8s. We gave the collector instance more memory, but we continued to see these errors
{"caller":"exporterhelper/queue_sender.go:92", "data_type":"metrics", "dropped_items":14317, "error":"not retryable error: Permanent error: rpc error: code = ResourceExhausted desc = limit exceeded", "kind":"exporter", "level":"error", "msg":"Exporting failed. Dropping data.", "name":"otlp", "stacktrace":"<http://go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1|go.opentelemetry.io/collector/exporter/exporterhelper.newQueueSender.func1>
	<http://go.opentelemetry.io/collector/exporter@v0.109.0/exporterhelper/queue_sender.go:92|go.opentelemetry.io/collector/exporter@v0.109.0/exporterhelper/queue_sender.go:92>
<http://go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Consume|go.opentelemetry.io/collector/exporter/internal/queue.(*boundedMemoryQueue[...]).Consume>
	<http://go.opentelemetry.io/collector/exporter@v0.109.0/internal/queue/bounded_memory_queue.go:52|go.opentelemetry.io/collector/exporter@v0.109.0/internal/queue/bounded_memory_queue.go:52>
<http://go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1|go.opentelemetry.io/collector/exporter/internal/queue.(*Consumers[...]).Start.func1>
	<http://go.opentelemetry.io/collector/exporter@v0.109.0/internal/queue/consumers.go:43|go.opentelemetry.io/collector/exporter@v0.109.0/internal/queue/consumers.go:43>", "ts":1.7326457593054848E9}
I can share our configuration as well in thread
Hi all, this is my first attempt at SigNoz. I'm running on Ubuntu, and I've used the installer scrip...
e

Erik Pragt

about 1 year ago
Hi all, this is my first attempt at SigNoz. I'm running on Ubuntu, and I've used the installer script, but no joy:
webserver@webserver:~/signoz/deploy$ ./install.sh

šŸ‘‹ Thank you for trying out SigNoz!

🟔 Running installer with non-sudo permissions.
   In case of any failure or prompt, please consider running the script with sudo privileges.

šŸŒ Detecting your OS ...

🐳 Starting Docker ...



🟔 Pulling the latest container images for SigNoz.

ERROR: The Compose file '././docker/clickhouse-setup/docker-compose.yaml' is invalid because:
services.otel-collector.depends_on.otel-collector-migrator.condition contains "service_completed_successfully", which is an invalid type, it should be a service_started, or a service_healthy
services.query-service.depends_on.otel-collector-migrator.condition contains "service_completed_successfully", which is an invalid type, it should be a service_started, or a service_healthy
šŸ”“ The containers didn't seem to start correctly. Please run the following command to check containers that may have errored out:

 docker-compose -f ./docker/clickhouse-setup/docker-compose.yaml ps -a
Please read our troubleshooting guide <https://signoz.io/docs/install/troubleshooting/>
or reach us for support in #help channel in our Slack Community <https://signoz.io/slack>
++++++++++++++++++++++++++++++++++++++++

šŸ“Ø Please share your email to receive support with the installation
I've attempted the troubleshooting guide, but I can't run something like this it seems:
sudo docker-compose -f ./docker/clickhouse-setup/docker-compose.yaml ps -a
the output is:
webserver@webserver:~/signoz/deploy$  docker-compose -f ./docker/clickhouse-setup/docker-compose.yaml ps -a
List containers.

Usage: ps [options] [SERVICE...]

Options:
    -q, --quiet          Only display IDs
    --services           Display services
    --filter KEY=VAL     Filter services by a property
Anyone a suggestion what I could try?
Hi, I'm starting up my microservice and the service is up and running but I dont see this service un...
p

Prajyod Arudra

about 1 year ago
Hi, I'm starting up my microservice and the service is up and running but I dont see this service under Services tab on Signoz. OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:3301/" \ OTEL_RESOURCE_ATTRIBUTES=service.name=pgm_migration \ java -javaagent:/Users/parudr920/git/iddl-pgm-migration/opentelemetry-javaagent.jar \ -Dapplication.rds.jdbcUrl=jdbc:mysql://localhost:5000 \ -Dapplication_port=8096 \ -Deureka.client.enabled=false \ -Dlogging.level.ROOT=INFO \ -Dspring-boot.run.jvmArguments="-Xms2048m -Xmx4096m" \ -Dspring.profiles.active=dev,kafka,swagger,mysql,iddlvault \ -jar /Users/parudr920/git/iddl-pgm-migration/target/iddl-pgm-migration-0.0.1-SNAPSHOT.jar Below are the errors I see. [otel.javaagent 2024-06-24 115515:748 -0600] [OkHttp http://localhost:3301/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export logs. Server responded with HTTP status code 405. Error message: Unable to parse response body, HTTP status message: Not Allowed [otel.javaagent 2024-06-24 115517:409 -0600] [OkHttp http://localhost:3301/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export spans. Server responded with HTTP status code 405. Error message: Unable to parse response body, HTTP status message: Not Allowed 2024-06-24 115517.422 WARN 78775 --- [ main] iguration$LoadBalancerCaffeineWarnLogger : Spring Cloud LoadBalancer is currently working with the default cache. You can switch to using Caffeine cache, by adding it and org.springframework.cache.caffeine.CaffeineCacheManager to the classpath. 2024-06-24 115517.555 INFO 78775 --- [ main] c.c.d.i.m.IddlPgmMigrationApplication : Started IddlPgmMigrationApplication in 133.944 seconds (JVM running for 135.639) [otel.javaagent 2024-06-24 115517:763 -0600] [OkHttp http://localhost:3301/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export logs. Server responded with HTTP status code 405. Error message: Unable to parse response body, HTTP status message: Not Allowed [otel.javaagent 2024-06-24 115602:233 -0600] [OkHttp http://localhost:3301/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export metrics. Server responded with HTTP status code 405. Error message: Unable to parse response body, HTTP status message: Not Allowed [otel.javaagent 2024-06-24 115702:237 -0600] [OkHttp http://localhost:3301/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export metrics. Server responded with HTTP status code 405. Error message: Unable to parse response body, HTTP status message: Not Allowed [otel.javaagent 2024-06-24 115802:251 -0600] [OkHttp http://localhost:3301/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export metrics. Server responded with HTTP status code 405. Error message: Unable to parse response body, HTTP status message: Not Allowed [otel.javaagent 2024-06-24 115902:233 -0600] [OkHttp http://localhost:3301/...] WARN io.opentelemetry.exporter.internal.http.HttpExporter - Failed to export metrics. Server responded with HTTP status code 405. Error message: Unable to parse response body, HTTP status message: Not Allowed any help is appreciated . Thanks