k8s_pod_phase=\"Pending\", k8s_pod_ready=\"false\"

^ that's alarming to me at first glance. Are all your pods coming up and resolving into a healthy state?

Hi, I want to scrape custom application metrics from their service endpoints with signoz. I am using service monitors CRD to do this with Prometheus currently. I don't want my Otel agent to scrape this Prometheus for the metrics. I want it to directly scrape the app endpoints attached to the service. Can I configure my Otel agent to collect these data? these are my requirements. 1. the scrape should not happen on the service endpoint instead it should hit the pod IP directly and the metrics collected should have the pod name label (similar to the service monitors implementation in Prometheus) 2. The scrape endpoints may change depending on the service. I want to have a service discovery mechanism for scrape config. Any tips or insights would be much appreciated!

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

sup yall, I'm trying to follow this guide and I get the following error from otelcol-contrib. has anyone seen this before? https://signoz.io/docs/tutorial/opentelemetry-binary-usage-in-virtual-machine/ error:

2024-12-24T10:58:18.630-0500	error	exporterhelper/retry_sender.go:145	Exporting failed. The error is not retryable. Dropping data.	{"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "Permanent error: rpc error: code = Unimplemented desc = unknown service opentelemetry.proto.collector.metrics.v1.MetricsService", "dropped_items": 2456}
<http://go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*retrySender).send>
	<http://go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/retry_sender.go:145|go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/retry_sender.go:145>
<http://go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send|go.opentelemetry.io/collector/exporter/exporterhelper.(*metricsSenderWithObservability).send>
	<http://go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/metrics.go:176|go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/metrics.go:176>
<http://go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1|go.opentelemetry.io/collector/exporter/exporterhelper.(*queueSender).start.func1>
	<http://go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/queue_sender.go:126|go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/queue_sender.go:126>
<http://go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1|go.opentelemetry.io/collector/exporter/exporterhelper/internal.(*boundedMemoryQueue).Start.func1>
	<http://go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/internal/bounded_memory_queue.go:52|go.opentelemetry.io/collector/exporter@v0.88.0/exporterhelper/internal/bounded_memory_queue.go:52>

hello one more im really look for some quick help on I am trying to run signoz on an ec2 instance in an aws account. setting up the service on the instance with isntall script has been working well enough. I am trying to monitor my apps/services in some ecs clusters. There are two clusters 1. is keycloak, and their new v26 build has an otel preview feature that works pretty well for tracing/metrics and uses grpc by default 2. our app stack has some sorting out to do before grpc, but we can get it to work with http endpoint. when i combine the collector to an ecs sidecar I can get it working with keycloak and grpc but I can't do the same for our app with http. Im using separate sidecar collector files swapping to port 4318 etc. seeing errors like

2024-12-04T20:32:09.836Z info exporterhelper/retry_sender.go:177 Exporting failed. Will retry the request after interval. {"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"error reading server preface: http2: frame too large\"", "interval": "6.306292489s"}

and can't track down why in thread is my yaml config file for the signoz sidecar for that app and below that are the variables for the cont def for our app I am trying to point it to. What am I doing wrong?

I am not able to fetch the logs of my docker container, it is not showing in the screen