This message was deleted.

Hi, my current setup looks like this 1. A self hosted signoz cluster running on K8s with helmchat (signoz/signoz) 2. A k8s cluster where all my work loads are running, its pushing data to the self hosted and its working perfectly fine 3. I tried to onboard another cluster today. but i keep getting this error in agent logs

{"level":"error","ts":1723615085.4947934,"caller":"scraperhelper/scrapercontroller.go:200","msg":"Error scraping metrics","kind":"receiver","name":"hostmetrics","data_type":"metrics","error":"failed to read usage at /hostfs/var/vcap/data/nsx-kube-proxy/rootfs/etc/hosts: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-kube-proxy/rootfs/etc/hostname: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-kube-proxy/rootfs/usr/lib/os-release: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-kube-proxy/rootfs/var/log/nsx-ujo: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-kube-proxy/rootfs/etc/nsx-ujo/ncp.ini: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-kube-proxy/rootfs/etc/nsx-ujo/certs: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-node-agent/rootfs/etc/hosts: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-node-agent/rootfs/etc/hostname: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-node-agent/rootfs/usr/lib/os-release: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-node-agent/rootfs/var/log/nsx-ujo: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-node-agent/rootfs/etc/nsx-ujo/ncp.ini: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-node-agent/rootfs/etc/nsx-ujo/certs: permission denied; failed to read usage at /hostfs/var/vcap/data/nsx-node-agent/rootfs/var/vcap/data/garden-cni/container-netns: permission denied","scraper":"filesystem","stacktrace":"<http://go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport|go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport>\n\<http://tgo.opentelemetry.io/collector/receiver@v0.88.0/scraperhelper/scrapercontroller.go:200|tgo.opentelemetry.io/collector/receiver@v0.88.0/scraperhelper/scrapercontroller.go:200>\<http://ngo.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1|ngo.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1>\n\<http://tgo.opentelemetry.io/collector/receiver@v0.88.0/scraperhelper/scrapercontroller.go:176|tgo.opentelemetry.io/collector/receiver@v0.88.0/scraperhelper/scrapercontroller.go:176>"}

4. Then i noticed that deployment.envrionmental stopped working too, while checking through, i decided to downgrade the helm chart to version 0.11.7

helm upgrade --namespace=platform my-release signoz/k8s-infra -f vgfacematch2.yaml --version 0.11.7

everything now seem to work. what could be the issue here? how do i get to use the latest version 0.11.9. i could not find any documentation maybe i'm missing a step

Hi , i tried signoz self-hosted (docker), im using nestjs, but i cant see any thing in my signoz. // tracer.ts 'use strict'; import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node'; import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http'; import { Resource } from '@opentelemetry/resources'; import { formattedLogger } from './utils/formatedLogger'; import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions'; import { ConfigService } from '@nestjs/config'; import { NodeSDK } from '@opentelemetry/sdk-node'; const logger = formattedLogger('Signoz'); /* const configService = new ConfigService(); const TRACE_ENABLED = configService.getOrThrow<boolean>('TRACE_ENABLED'); logger.debug('TRACE_ENABLED: ' + TRACE_ENABLED); if (!TRACE_ENABLED) { return; } */ // Configure the SDK to export telemetry data to SigNoz const exporterOptions = { url: 'http://adelpro.duckdns.org:4318/v1/traces', }; const traceExporter = new OTLPTraceExporter(exporterOptions); const sdk = new NodeSDK({ traceExporter, instrumentations: [ getNodeAutoInstrumentations({ '@opentelemetry/instrumentation-nestjs-core': { enabled: true }, }), ], resource: new Resource({ [SemanticResourceAttributes.SERVICE_NAME]: 'imcoder-backend', }), }); sdk.start(); // Gracefully shut down the SDK on process exit process.on('SIGTERM', () => { sdk .shutdown() .then(() => logger.log('Tracing terminated')) .catch((error) => logger.error('Error terminating tracing: ' + error)) .finally(() => process.exit(0)); }); process.on('SIGINT', () => { sdk .shutdown() .then(() => logger.log('Tracing terminated')) .catch((error) => logger.error('Error terminating tracing:' + error)) .finally(() => process.exit(0)); }); export default sdk; // main.ts ... tracer.start() ...

How to get nested spans for a request?

const trace = signozTracer.startSpan(functionName);
const span = trace.addEvent('mongo-getbyid');

I tried the above approach, but I am getting

mongo-getbyid

in events section, instead of in spans below main span. Any advice to manually instrument database performance traces so that I can see it in a request span as well as separately in mongo group for example.

Hey folks. I've set up Signoz twice - once with EKS and nginx-ingress-4.6.1 and cert-manager-1.12.3, and then with K3s on AWS with nginx-ingress-4.3.0 and cert-manager-1.9.1(the versions bundled with the signoz chart). I've provisioned certificates via LetsEncrypt and exposed the collector endpoint publicly - but the

k8s-infra

agent deployment seems unable to push data. Keeps failing with a handshake error:

2024-02-02T05:37:36.132Z        info    exporterhelper/retry_sender.go:177      Exporting failed. Will retry the request after interval.        {"kind": "exporter", "data_type": "metrics", "name": "otlp", "error": "rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: remote error: tls: handshake failure\"", "interval": "5.08956352s"}

. I can confirm that in both clusters, ingress and TLS work fine(since the Signoz frontend's also deployed and is easily accessible. I've tried adding the contents of

tls.crt

and

tls.key

from the certificate

Secret

on the SIgnoz cluster, to the Helm values for

k8s-infra

with the

otelTlsSecrets

block.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.

This message was deleted.