https://signoz.io logo
Powered by
#support
  • m

    Mathias Wegner

    01/10/2022, 4:31 PM
    I've set up signoz on my macbook to run a small poc. I took a basic django app (essentially the polls app in the django tutorial), and while I can successfully get all of the api calls, I'm unable to collect any database calls. I'm using the most recent commit to the signoz repo with docker and clickhouse, and I started my django app with
    Copy code
    ~/src/apm-poc/example> OTEL_RESOURCE_ATTRIBUTES=service.name=example OTEL_EXPORTER_OTLP_ENDPOINT="<http://127.0.0.1:4317>" OTEL_LOG_LEVEL=debug opentelemetry-instrument python3 manage.py runserver --noreload
Performing system checks...

System check identified no issues (0 silenced).
January 10, 2022 - 11:22:20
Django version 4.0.1, using settings 'example.settings'
Starting development server at <http://127.0.0.1:8000/>
Quit the server with CONTROL-C.
[10/Jan/2022 11:22:21] "GET /admin/ HTTP/1.1" 302 0
[10/Jan/2022 11:22:21] "GET /admin/login/?next=/admin/ HTTP/1.1" 200 2211
[10/Jan/2022 11:22:26] "POST /admin/login/?next=/admin/ HTTP/1.1" 302 0
[10/Jan/2022 11:22:26] "GET /admin/ HTTP/1.1" 200 3974
[10/Jan/2022 11:22:26] "GET /static/admin/css/dashboard.css HTTP/1.1" 304 0
[10/Jan/2022 11:22:26] "GET /static/admin/img/icon-addlink.svg HTTP/1.1" 304 0
[10/Jan/2022 11:22:26] "GET /static/admin/img/icon-changelink.svg HTTP/1.1" 304 0
    p
    a
    • 3
    • 16
  • e

    edr

    01/13/2022, 7:22 AM
    Hi all, is there an easy way to configure a different network for the docker compose tied to signoz?
    a
    p
    • 3
    • 8
  • v

    Vinícius Costa

    01/13/2022, 2:01 PM
    Are there any guides or use cases of using SignNoz with AWS? I would like to collect some logs from cloudwatch there
    p
    a
    • 3
    • 4
  • s

    Sagar Dutta

    01/17/2022, 10:13 AM
    2022/01/17 072950 [warn] 1#1: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2 nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2 2022/01/17 072950 [emerg] 1#1: mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied) nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (13: Permission denied)
    a
    p
    • 3
    • 14
  • s

    Sagar Dutta

    01/17/2022, 10:14 AM
    2022-01-17T101224.871Z [35mDEBUG[0m query-service/main.go:31 START! 2022-01-17T101224.872Z [34mINFO[0m app/server.go:109 Using ClickHouse as datastore ... 2022-01-17T101229.872Z [31mERROR[0m clickhouseReader/reader.go:90 error connecting to primary db: dial tcp 172.30.57.2369000 i/o timeout go.signoz.io/query-service/app/clickhouseReader.NewReader /go/src/github.com/signoz/signoz/pkg/query-service/app/clickhouseReader/reader.go:90 go.signoz.io/query-service/app.createHTTPServer /go/src/github.com/signoz/signoz/pkg/query-service/app/server.go:110 go.signoz.io/query-service/app.NewServer /go/src/github.com/signoz/signoz/pkg/query-service/app/server.go:69 main.main /go/src/github.com/signoz/signoz/pkg/query-service/main.go:41 runtime.main /usr/local/go/src/runtime/proc.go:203
    a
    p
    • 3
    • 4
  • s

    Sagar Dutta

    01/17/2022, 10:54 AM
    securityContext:         capabilities:           drop:             - KILL             - MKNOD             - SETGID             - SETUID         runAsUser: 1000850000
    a
    • 2
    • 1
  • d

    Ditty K.M

    01/17/2022, 5:06 PM
    We are sending telemetry data to SigNoz using PL/Python function inside PostgreSQL. But sometimes we can't get the data to SigNoz. Even if this data reaches SigNoz, there will be a terrible delay. If we send data from outside PostgreSQL its working perfectly. Is this due to any specific python version requirement? Please advise me for the same, we can't figure out the issue. We are using 4317 port for sending data
    p
    • 2
    • 2
  • r

    Ranjit Barsa

    01/24/2022, 7:31 AM
    hello team can anyone help me on there's no data showing in metrics I have deployed signoz in local docker but when I am accessing http://localhost:3000 it's showing No Data
    p
    • 2
    • 8
  • r

    Ranjit Barsa

    01/24/2022, 7:32 AM
    and when I deployed signoz on server it shows below data
    a
    • 2
    • 8
  • m

    Martin Pola

    01/26/2022, 5:56 PM
    Hello! I just installed Signoz and I'm trying to figure out what's the best way to monitor CPU usage (%), disk usage, and other system-wide metrics for my servers. Is that possible with Signoz, and if so, how would I go about? I'm thinking that I'll need to install some sort of agent on each server, and point them to my Signoz backend/instance.
    p
    • 2
    • 2
  • o

    Onur Yikilmazoglu

    01/27/2022, 1:06 PM
    I'm trying to run the signoz without containerization, I started out with trying to run the frontend outside the container and the rest in the container, however even when point baseURL to localhost:8080 I'm getting just a blank screen. what am I doing wrong?
    p
    p
    a
    • 4
    • 19
  • p

    Prabhu Chawandi

    01/30/2022, 6:19 AM
    Hello, currently, we our server is run with docker-compose, and we are using fluentd to collect all the logs from each server instance and push it to the log management server that we have set up. So, I was thinking to use sigNoz here, so what is the ideal way that we start using the SignNoz for our setup.
    p
    • 2
    • 4
  • o

    Onur Yikilmazoglu

    01/31/2022, 2:35 PM
    I've got everything working since my last post here, however I have one big issue, has anyone figured out some sort of login, authentication or anything that wont let anyone use the frontend
    p
    • 2
    • 9
  • j

    Jingqi Huang

    02/02/2022, 7:08 PM
    I run 
    kubectl describe pvc
    for all the pvc and I got the same message 
    FailedBinding  23s (x903 over 3h45m)  persistentvolume-controller  no persistent volumes available for this claim and no storage class is set
    p
    • 2
    • 7
  • j

    Jingqi Huang

    02/02/2022, 7:09 PM
    Besides install signoz/signoz as suggested in https://signoz.io/docs/deployment/helm_chart, I am wondering what extra steps I need to take to make all the pods run. Thanks!
    a
    • 2
    • 1
  • t

    Thiago Alexandria

    02/02/2022, 8:44 PM
    Hi team, I'm trying to run signoz locally on my Mac (intel chip), using dockerd through rancher desktop. I'm having trouble because my clickhouse-server doesn't start, I get the following error from the container:
    Copy code
    Poco::Exception. Code: 1000, e.code() = 0, Not found: user_files_path (version 21.10.5.3 (official build))
Processing configuration file '/etc/clickhouse-server/config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Include not found: clickhouse_remote_servers
Include not found: clickhouse_compression
Logging trace to /var/log/clickhouse-server/clickhouse-server.log
Logging errors to /var/log/clickhouse-server/clickhouse-server.err.log
Processing configuration file '/etc/clickhouse-server/config.xml'.
Merging configuration file '/etc/clickhouse-server/config.d/docker_related_config.xml'.
Include not found: clickhouse_remote_servers
Include not found: clickhouse_compression
Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/config.xml'.
Processing configuration file '/etc/clickhouse-server/users.xml'.
Saved preprocessed configuration to '/var/lib/clickhouse/preprocessed_configs/users.xml'.
ClickHouse init process failed.
    I looked for the chat and I couldn't find something that helps me, has anyone here gone through this?
    p
    p
    • 3
    • 5
  • o

    Onur Yikilmazoglu

    02/03/2022, 6:06 PM
    I'm having a few problems with the otel collector, not sure if I should ask here or somewhere else since its not exactly what you guys do, however I keep having this problem where there the otel collector will balloon in ram usage and will eat most of the ram and almost all of my swap. I just check the my server and it using 6.5 gb of swap alone. I killed it with htop and swap usage dropped back down to 190 mb. However this is at best temporary, has anyone else ran into this problem?
    p
    a
    • 3
    • 19
  • p

    Prabhu Chawandi

    02/07/2022, 5:11 AM
    Where I'm doing wrong? Could you guide me please?
    p
    a
    • 3
    • 4
  • p

    Patrik Potocki

    02/09/2022, 12:56 PM
    Is there any documentation around how the Legend Format works in Metrics dashboard? 🙂
    a
    • 2
    • 2
  • p

    Patrik Potocki

    02/09/2022, 1:36 PM
    Questions , is there a reason why this is hardcoded? https://github.com/SigNoz/signoz/blob/28bf2fe3f7c0b013685921c9d4bfdb254ac42e81/pkg/query-service/constants/constants.go#L15 ? It's causes probems when I used the helm chart to install cause then my alertmanager service will be on 
    <http://signoz-alertmanager:9093/api/>
    ? 🙂
    a
    • 2
    • 4
  • p

    Patrik Potocki

    02/10/2022, 9:10 AM
    Is there a way to see what "metrics" you have available? cause quite hard to build those dashboard without knowing all metrics available. For example I tried to find what metrics you get from the 
    kubeletstats
    collector
    ✅ 1
    p
    a
    • 3
    • 9
  • d

    Daniel Lima

    02/10/2022, 2:41 PM
    p
    a
    • 3
    • 99
  • s

    Selva

    02/12/2022, 3:34 PM
    Hi I’m trying to create node js micro service with mysql db call, when i try do an api call to my endpoints able to see those in metrics, whereas the mysql db calls are not getting populated in Database Calls tab, can someone help?
    p
    a
    • 3
    • 14
  • s

    Selva

    02/14/2022, 11:10 AM
    why console.errors are not captured here? on the right end Details for selected Span which type of error can we capture here can someone help?
    p
    • 2
    • 1
  • p

    praddy

    02/15/2022, 5:35 AM
    Hello folks On executing command 
    go get <http://go.signoz.io/query-service/version|go.signoz.io/query-service/version>
    I am getting following error 
    go get: unrecognized import path "<http://go.signoz.io/query-service/version|go.signoz.io/query-service/version>": https fetch: Get "<https://go.signoz.io/query-service/version?go-get=1>": x509: certificate has expired or is not yet valid: current time 2022-02-15T11:01:04+05:30 is after 2021-06-12T15:18:58Z
    Can someone please provide inputs here
    p
    v
    • 3
    • 2
  • n

    Nico Van Wyk

    02/16/2022, 12:16 PM
    Copy code
    apiVersion: <http://traefik.containo.us/v1alpha1|traefik.containo.us/v1alpha1>
kind: IngressRoute
metadata:
  annotations:
    <http://kubernetes.io/ingress.class|kubernetes.io/ingress.class>: traefik
  name: signoz
spec:
  entryPoints:
    - web
  routes:
    - kind: Rule
      match: Host(`tools.localhost`) && PathPrefix(`/signoz`)
      middlewares:
        - name: signoz
      services:
        - name: signoz-frontend
          port: 3301

---

apiVersion: <http://traefik.containo.us/v1alpha1|traefik.containo.us/v1alpha1>
kind: Middleware
metadata:
  name: signoz
spec:
  headers:
    customRequestHeaders:
      X-Script-Name: "signoz"
    p
    p
    • 3
    • 17
  • a

    Axay Sagathiya

    02/18/2022, 5:42 AM
    Hello folks, I'm new to the open-source world and want to contribute to the backend. all the steps given in this documentation are necessary to set up a local development environment?
    p
    v
    • 3
    • 46
  • k

    Karam Ashqar

    02/18/2022, 11:44 PM
    hi, I installed everything on EKS, attached a Loadbalancer nginx to my-release-signoz-frontend svc, created account, logged in, then I attached another Loadbalancer nginx to my-release-signoz-otel-collector mapped to port 55681 I can see this response when I call <loadbalancer_externalip>/v1/trace
    Copy code
    {
code: 12,
message: "Not Implemented",
details: [ ]
}
    and when I put this link to my nestjs app on my local machine or cluster, it doesn't show any metrics/applications at all, anyone knows why?
    p
    • 2
    • 11
  • p

    paz.lucky

    02/21/2022, 1:10 AM
    The signoz docker installation fails. panic: runtime error: invalid memory address or nil pointer reference problem occurs.
    p
    • 2
    • 16
  • r

    Rajneesh Mehta

    02/22/2022, 9:30 PM
    getting 
    Overriding of current TracerProvider is not allowed
    and application is not sending any data to signoz backend. Here's what configuration I've done. installed on: 
    minikube
    installed with: 
    helm
    release name: 
    signoz-uat
    backend framework: 
    django
    running server with : 
    gunicorn
    command: 
    OTEL_METRICS_EXPORTER=none DJANGO_SETTINGS_MODULE=shopfloor.settings  OTEL_RESOURCE_ATTRIBUTES=service.name=stitch OTEL_EXPORTER_OTLP_ENDPOINT="signoz-uat-otel-collector.platform.svc.cluster.local:4317" opentelemetry-instrument gunicorn shopfloor.wsgi -c gunicorn.config.py --bind 0.0.0.0:8000
    content of 
    gunicorn.config.py
    Copy code
    from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor


def post_fork(server, worker):
    <http://server.log.info|server.log.info>("Worker spawned (pid: %s)", worker.pid)

    resource = Resource.create(attributes={
        "service.name": "stitch"
    })

    trace.set_tracer_provider(TracerProvider(resource=resource))
    span_processor = BatchSpanProcessor(
        OTLPSpanExporter(endpoint="signoz-uat-otel-collector.platform.svc.cluster.local:4317")
    )
    trace.get_tracer_provider().add_span_processor(span_processor)
    result from running Troubleshooting binary:-
    Copy code
    root@django-backend-deployment-bb8875d7c-h9v4l:/app# ./troubleshoot checkEndpoint --endpoint=signoz-uat-otel-collector.platform.svc.cluster.local:4317
2022-02-22T21:27:43.780Z        INFO    workspace/main.go:28    STARTING!
2022-02-22T21:27:43.780Z        INFO    checkEndpoint/checkEndpoint.go:41       checking reachability of SigNoz endpoint
2022-02-22T21:27:43.796Z        INFO    workspace/main.go:46    Successfully sent sample data to signoz ...
    this is what 
    <http://localhost:8888/metrics> contains
    Copy code
    # HELP otelcol_exporter_queue_size Current size of the retry queue (in batches)
# TYPE otelcol_exporter_queue_size gauge
otelcol_exporter_queue_size{exporter="clickhousemetricswrite",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 0
# HELP otelcol_exporter_send_failed_metric_points Number of metric points in failed attempts to send to destination.
# TYPE otelcol_exporter_send_failed_metric_points counter
otelcol_exporter_send_failed_metric_points{exporter="prometheus",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 0
# HELP otelcol_exporter_send_failed_spans Number of spans in failed attempts to send to destination.
# TYPE otelcol_exporter_send_failed_spans counter
otelcol_exporter_send_failed_spans{exporter="clickhouse",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 0
# HELP otelcol_exporter_sent_metric_points Number of metric points successfully sent to destination.
# TYPE otelcol_exporter_sent_metric_points counter
otelcol_exporter_sent_metric_points{exporter="prometheus",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 0
# HELP otelcol_exporter_sent_spans Number of spans successfully sent to destination.
# TYPE otelcol_exporter_sent_spans counter
otelcol_exporter_sent_spans{exporter="clickhouse",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 3
# HELP otelcol_process_cpu_seconds Total CPU user and system time in seconds
# TYPE otelcol_process_cpu_seconds gauge
otelcol_process_cpu_seconds{service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 15.25
# HELP otelcol_process_memory_rss Total physical memory (resident set size)
# TYPE otelcol_process_memory_rss gauge
otelcol_process_memory_rss{service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 1.3193216e+08
# HELP otelcol_process_runtime_heap_alloc_bytes Bytes of allocated heap objects (see 'go doc runtime.MemStats.HeapAlloc')
# TYPE otelcol_process_runtime_heap_alloc_bytes gauge
otelcol_process_runtime_heap_alloc_bytes{service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 4.912016e+07
# HELP otelcol_process_runtime_total_alloc_bytes Cumulative bytes allocated for heap objects (see 'go doc runtime.MemStats.TotalAlloc')
# TYPE otelcol_process_runtime_total_alloc_bytes gauge
otelcol_process_runtime_total_alloc_bytes{service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 2.645922376e+09
# HELP otelcol_process_runtime_total_sys_memory_bytes Total bytes of memory obtained from the OS (see 'go doc runtime.MemStats.Sys')
# TYPE otelcol_process_runtime_total_sys_memory_bytes gauge
otelcol_process_runtime_total_sys_memory_bytes{service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 8.30064824e+08
# HELP otelcol_process_uptime Uptime of the process
# TYPE otelcol_process_uptime counter
otelcol_process_uptime{service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 7485.002079331997
# HELP otelcol_processor_batch_batch_send_size Number of units in the batch
# TYPE otelcol_processor_batch_batch_send_size histogram
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="10"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="25"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="50"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="75"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="100"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="250"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="500"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="750"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="1000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="2000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="3000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="4000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="5000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="6000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="7000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="8000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="9000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="10000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="20000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="30000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="50000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="100000"} 3
otelcol_processor_batch_batch_send_size_bucket{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",le="+Inf"} 3
otelcol_processor_batch_batch_send_size_sum{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 3
otelcol_processor_batch_batch_send_size_count{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 3
# HELP otelcol_processor_batch_timeout_trigger_send Number of times the batch was sent due to a timeout trigger
# TYPE otelcol_processor_batch_timeout_trigger_send counter
otelcol_processor_batch_timeout_trigger_send{processor="batch",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c"} 3
# HELP otelcol_receiver_accepted_spans Number of spans successfully pushed into the pipeline.
# TYPE otelcol_receiver_accepted_spans counter
otelcol_receiver_accepted_spans{receiver="otlp",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",transport="grpc"} 3
# HELP otelcol_receiver_refused_spans Number of spans that could not be pushed into the pipeline.
# TYPE otelcol_receiver_refused_spans counter
otelcol_receiver_refused_spans{receiver="otlp",service_instance_id="0d36facb-49c4-4315-af49-10bea6e2c55c",transport="grpc"} 0
    Can you please help what should I do?
    a
    • 2
    • 8
12345...55Latest