https://signoz.io logo
Join the conversationJoin Slack
Channels
contributing
contributing-frontend
general
github-activities
incidents
k8s_operator_helm
reactjs-config
signoz-alert-testing
support
testsupport
watercooler
webhook-dev
write-for-signoz
Powered by Linen
support
  • s

    Siddhant Sahay

    08/05/2022, 11:49 AM
    Hello! I was setting up signoz for monitoring my rust services. Need some help in integrating opentelemetry into warp(rust server) https://github.com/seanmonstar/warp.
    s
    n
    • 3
    • 7
  • p

    Patrick Sauerborn

    08/06/2022, 7:07 PM
    Hi everybody, has anybody created a Discord webhook alert channel yet? I tried to create one and when I tested it, I got a 500 error. Some information about my setup: I hosted SigNoz on my private server, which is running Linux. The only open ports are for http, https and ssh (typical webserver setup). SigNoz is running on a Docker container, which is accessible via proxy behind a nginx webserver. The exposed ports of Docker are only accessible on localhost (if that matters). I would be glad, if anybody has an idea, what could cause this behavior. Best regards, Patrick
    s
    p
    • 3
    • 3
  • d

    David Castillo

    08/08/2022, 8:27 AM
    Hi! I just deployed to GCP, followed the docs exactly, everything seems to be running fine but I'm not allowed to register for the first time. I always get an authetication error when registering. Any ideas?
    p
    a
    • 3
    • 7
  • a

    Anil Kumar Bandrapalli

    08/08/2022, 9:24 AM
    Hi @Ankit Nayan, we are trying to fetch the mongodb traces from a node js application request. but we are unable to get that info from node js http request trace. is there any way to get traces of mongodb ? We are using mongoose for the connecting with db. is it causing traces not to visible on signoz ?
    a
    a
    +3
    • 6
    • 157
  • a

    Alexei Zenin

    08/08/2022, 6:02 PM
    I upgraded from 0.10 query service and frontend to 0.10.1 and now I get this in the query service logs:
    Table signoz_traces.top_level_operations doesn't exist
    Did I miss updating the collector, didn’t see anything mentioned in the release notes, wish the query service would be the component that handles schema creation instead of the collectors. Seems like the minor version bump might have had a breaking change if not aligned with the right collector?
    s
    p
    • 3
    • 21
  • w

    William Wang

    08/10/2022, 3:13 AM
    I need to reset password for signoz, who can help me to process, my email is kspine@163.com Thanks
    p
    p
    • 3
    • 13
  • u

    Upesh Jindal

    08/10/2022, 5:22 AM
    Hey everyone! I'm new to SigNoz and set it up today morning only.
  • u

    Upesh Jindal

    08/10/2022, 5:23 AM
    It's set up on FastAPI, Python 3.10 with postgres. I was able to see DB calls in the APIs this morning but now they have all disappeared.
  • u

    Upesh Jindal

    08/10/2022, 5:23 AM
    I read in the docs that psycopg2-binary is not supported by opentelemetry so need to use psycopg2. I did that but still no SQLs
  • u

    Upesh Jindal

    08/10/2022, 6:02 AM
    Answered it myself. Had to reinstall things and forgot
    opentelemetry-bootstrap --action=install
    🤦‍♂️
  • u

    Upesh Jindal

    08/10/2022, 6:24 AM
    Same is for Error percentage
  • s

    Srikanth Chekuri

    08/10/2022, 6:25 AM
    Can you share the request payload for this?
    u
    • 2
    • 12
  • u

    Upesh Jindal

    08/10/2022, 8:51 AM
    Has anyone faced the following exception:
    Traceback (most recent call last):
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/opentelemetry/sdk/metrics/_internal/instrument.py", line 132, in callback
        for api_measurement in callback(callback_options):
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/opentelemetry/instrumentation/system_metrics/__init__.py", line 563, in _get_system_network_connections
        for net_connection in psutil.net_connections():
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/__init__.py", line 2158, in net_connections
        return _psplatform.net_connections(kind)
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/_psosx.py", line 248, in net_connections
        cons = Process(pid).connections(kind)
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/_psosx.py", line 350, in wrapper
        raise AccessDenied(self.pid, self._name)
    psutil.AccessDenied: (pid=27843)
    Callback failed for instrument system.network.connections.
    Traceback (most recent call last):
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/_psosx.py", line 343, in wrapper
        return fun(self, *args, **kwargs)
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/_psosx.py", line 500, in connections
        rawlist = cext.proc_connections(self.pid, families, types)
    PermissionError: [Errno 1] Operation not permitted (originated from proc_pidinfo(PROC_PIDLISTFDS) 1/2)
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/opentelemetry/sdk/metrics/_internal/instrument.py", line 132, in callback
        for api_measurement in callback(callback_options):
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/opentelemetry/instrumentation/system_metrics/__init__.py", line 563, in _get_system_network_connections
        for net_connection in psutil.net_connections():
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/__init__.py", line 2158, in net_connections
        return _psplatform.net_connections(kind)
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/_psosx.py", line 248, in net_connections
        cons = Process(pid).connections(kind)
      File "/Users/upeshjindal/Virtual/healthpe.api.backend.py/lib/python3.10/site-packages/psutil/_psosx.py", line 350, in wrapper
        raise AccessDenied(self.pid, self._name)
    psutil.AccessDenied: (pid=27843)
    p
    s
    • 3
    • 7
  • u

    Upesh Jindal

    08/10/2022, 8:51 AM
    Searched for this in issues but didn't find any.
  • r

    Raghu

    08/11/2022, 1:04 AM
    👋 Hello, team!
  • r

    Raghu

    08/11/2022, 1:05 AM
    might be too basic for a support channel, but I remember reading it somewhere, but can't find it now. How do I login to signoz UI thats installed in a k8s cluster . Thanks
    p
    • 2
    • 4
  • u

    Upesh Jindal

    08/11/2022, 3:39 AM
    Hi, is there any way to ignore certain endpoints?
    p
    s
    a
    • 4
    • 16
  • u

    Upesh Jindal

    08/11/2022, 5:14 AM
    Is there a way to auto refresh the dashboard and also to keep the sorting fixed by a certain metric?
    a
    • 2
    • 3
  • r

    rakesh shah

    08/11/2022, 6:40 AM
    s3 storage is a cold storage, does not pull data from the clickhouse -> s3. Does any one has idea?
  • r

    rakesh shah

    08/11/2022, 6:42 AM
    Specifically - we slowly increase the load of micro-service data - but we are facing challenges - with the query builder and unable to load. - any idea or suggestion - or any pointer talks about the scalability of the system.
    s
    p
    a
    • 4
    • 25
  • u

    Upesh Jindal

    08/11/2022, 9:58 AM
    Can anyone direct me to a gallery of custom dashboards, if any
    p
    a
    • 3
    • 36
  • a

    Anant Vashishtha

    08/11/2022, 11:55 AM
    Hi, Is version 0.10.2 or 0.10.1 available on the main branch yet? I followed the steps to upgrade but it is still on 0.10.0
    p
    p
    • 3
    • 6
  • c

    Chris Ahern

    08/11/2022, 5:59 PM
    Hi, Signoz (in kubernetes) has been working well but has recently started giving OOM (out of memory) errors in the signoz-query-service. I've restarted the pod several times but has not helped. Here are the pods:
    kubectl -n platform get pods
    NAME                                                        READY   STATUS             RESTARTS   AGE
    my-release-zookeeper-0                                      1/1     Running            0          77d
    chi-signoz-cluster-0-0-0                                    1/1     Running            0          77d
    my-release-signoz-alertmanager-0                            1/1     Running            0          77d
    my-release-signoz-frontend-6b7dbccbc7-fgbnv                 1/1     Running            0          77d
    my-release-signoz-otel-collector-metrics-68bcfd5556-7tjks   1/1     Running            0          77d
    my-release-signoz-otel-collector-66c8c7dc9d-xqxbd           1/1     Running            0          77d
    clickhouse-operator-7f7f84b899-dcgs4                        2/2     Running            0          21h
    my-release-signoz-query-service-0                           0/1     CrashLoopBackOff   201        18h
    In the description of the my-release-signoz-query-service-0 pod, it shows:
    Containers:
      signoz:
        Container ID:  <containerd://aebb11f5525602fd8205e1aa49414724c6afc7c5b1784993bfe58d3e5a7e931>5
        Image:         <http://docker.io/signoz/query-service:0.8.0|docker.io/signoz/query-service:0.8.0>
        Image ID:      <http://docker.io/signoz/query-service@sha256:2febce16a8b8feb6bf96439eccccba4e6ab8e7ef401de96ea38fd1d42c3d9353|docker.io/signoz/query-service@sha256:2febce16a8b8feb6bf96439eccccba4e6ab8e7ef401de96ea38fd1d42c3d9353>
        Port:          8080/TCP
        Host Port:     0/TCP
        Args:
          -config=/root/config/prometheus.yml
        State:          Waiting
          Reason:       CrashLoopBackOff
        Last State:     Terminated
          Reason:       OOMKilled
          Exit Code:    137
          Started:      Thu, 11 Aug 2022 10:54:03 -0700
          Finished:     Thu, 11 Aug 2022 10:54:42 -0700
        Ready:          False
        Restart Count:  201
        Limits:
          cpu:     750m
          memory:  1000Mi
        Requests:
          cpu:      200m
          memory:   300Mi
    I would increase the memory limit but don't see a deployment for the my-release-signoz-query-service-0 pod:
    kubectl -n platform get deploy
    NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
    my-release-signoz-frontend                 1/1     1            1           77d
    my-release-signoz-otel-collector-metrics   1/1     1            1           77d
    my-release-signoz-otel-collector           1/1     1            1           77d
    clickhouse-operator                        1/1     1            1           77d
    How do I fix this?
    p
    p
    • 3
    • 5
  • a

    Aditya Tripathi

    08/12/2022, 8:08 AM
    Hey, trying to setup NestJS with OpenTel + Signoz but after following the sample repository, I encounter this error. Any clue what's going on there?
    credentials: grpc.credentials.createInsecure(),
                        ^
    TypeError: Cannot read properties of undefined (reading 'credentials')
        at Object.<anonymous> (/home/xxx/src/tracer.ts:17:21)
        at Module._compile (node:internal/modules/cjs/loader:1103:14)
        at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
        at Module.load (node:internal/modules/cjs/loader:981:32)
        at Function.Module._load (node:internal/modules/cjs/loader:822:12)
        at Module.require (node:internal/modules/cjs/loader:1005:19)
        at require (node:internal/modules/cjs/helpers:102:18)
        at Object.<anonymous> (/home/xxx/src/main.ts:13:1)
        at Module._compile (node:internal/modules/cjs/loader:1103:14)
        at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
    a
    p
    • 3
    • 14
  • p

    Pranay

    08/13/2022, 11:27 AM
    @Jatin Sharma you can just start the app again with appropriate steps to start sending data again
    • 1
    • 1
  • k

    Krishna Chaitanya V

    08/16/2022, 2:46 PM
    👋 Hello, team! I am currently using jaeger with open telemetry; I want to move to Signoz. Can someone help me out with how to do it?
    p
    • 2
    • 4
  • a

    Alexei Zenin

    08/17/2022, 3:47 PM
    I added service.namespace and deployment.environment to my process but don’t see the ability to filter by it in the UI (using 0.10.2). The process was running without those parameters before and sending to SigNoz. Is there something I need to do to enable it? Config for collector:
    receivers:
      otlp/spanmetrics:
        protocols:
          grpc:
            endpoint: localhost:12345
      otlp:
        protocols:
          http:
            endpoint: 0.0.0.0:4318
    
    processors:
      batch:
        send_batch_size: 10000
        send_batch_max_size: 11000
        timeout: 10s
      signozspanmetrics/prometheus:
        metrics_exporter: prometheus
        latency_histogram_buckets: [100us, 1ms, 2ms, 6ms, 10ms, 50ms, 100ms, 250ms, 500ms, 1000ms, 1400ms, 2000ms, 5s, 10s, 20s, 40s, 60s ]
        dimensions_cache_size: 10000
        dimensions:
          - name: service.namespace
            default: default
          - name: deployment.environment
            default: default
    
    exporters:
      clickhousetraces:
        datasource: tcp://${CLICKHOUSE_HOST}:${CLICKHOUSE_PORT}/?database=signoz_traces&username=${CLICKHOUSE_USER}&password=${CLICKHOUSE_PWD}
      clickhousemetricswrite:
        endpoint: tcp://${CLICKHOUSE_HOST}:${CLICKHOUSE_PORT}/?database=signoz_metrics&username=${CLICKHOUSE_USER}&password=${CLICKHOUSE_PWD}
        resource_to_telemetry_conversion:
          enabled: true
      prometheus:
        endpoint: 0.0.0.0:8889
    
    extensions:
      health_check:
        endpoint: 0.0.0.0:13133
    service:
      telemetry:
        metrics:
          address: 0.0.0.0:8888
      extensions: [health_check]
      pipelines:
        traces:
          receivers: [otlp]
          processors: [signozspanmetrics/prometheus, batch]
          exporters: [clickhousetraces]
        metrics:
          receivers: [otlp]
          processors: [batch]
          exporters: [clickhousemetricswrite]
        metrics/spanmetrics:
          receivers: [otlp/spanmetrics]
          exporters: [prometheus]
    Note: I am not sending directly to this collector but via an agent collector on the instance which forwards to this one via OTLP
    p
    p
    • 3
    • 12
  • m

    Mukesh Chaudhary

    08/19/2022, 5:34 AM
    Hello, I followed the instruction from docs to install signoz in kuberntes. I used different namespace (signoz) and different release name (signoz-dev). I installed the example also using this singoz example. Signoz has been installed properly. I just change values of ingress only in values.yaml file. But I am facing error
    something went wrong
    . The log of otel is below:
    2022-08-19T04:48:22.427Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
    <http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
    	/src/exporter/clickhousetracesexporter/writer.go:101
    2022-08-19T04:48:32.432Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
    <http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
    	/src/exporter/clickhousetracesexporter/writer.go:101
    2022-08-19T04:48:42.430Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
    <http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
    	/src/exporter/clickhousetracesexporter/writer.go:101
    2022-08-19T04:48:52.444Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
    <http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
    	/src/exporter/clickhousetracesexporter/writer.go:101
    2022-08-19T04:49:02.443Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
    <http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
    	/src/exporter/clickhousetracesexporter/writer.go:101
    2022-08-19T04:49:12.437Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
    <http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
    	/src/exporter/clickhousetracesexporter/writer.go:101
    2022-08-19T04:49:22.438Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
    <http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
    	/src/exporter/clickhousetracesexporter/writer.go:101
    Logs
    Also in UI there is no metrics section.
    p
    • 2
    • 18
  • u

    Upesh Jindal

    08/21/2022, 5:33 AM
    hi, is there any way I can see the infrastructure? For instance, how many API servers I have running
    p
    s
    • 3
    • 21
  • b

    Blake Romano

    08/23/2022, 9:36 PM
    I’d like to update my collector to 0.58.0, how badly will clickhouse break? https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/tag/v0.58.0
    s
    p
    • 3
    • 8
Powered by Linen
Title
b

Blake Romano

08/23/2022, 9:36 PM
I’d like to update my collector to 0.58.0, how badly will clickhouse break? https://github.com/open-telemetry/opentelemetry-collector-contrib/releases/tag/v0.58.0
s

Srikanth Chekuri

08/24/2022, 2:53 AM
Clickhouse and schemas are independent of collector releases.
p

Pranay

08/24/2022, 2:59 AM
@Blake Romano Curious, what features are there in 0.58.0 because of which you want to upgrade?
b

Blake Romano

08/24/2022, 12:54 PM
Because I like trying to keep Operator and Collector versions in sync, Operator released .Net autoinstrumentation support and also keeping things up to date is always good 🙂
Fair to say that updating to 0.58 will work fine tho?
s

Srikanth Chekuri

08/24/2022, 9:49 PM
Fair to say that updating to 0.58 will work fine tho?
Yes, I think it requires very minor changes (declaring stability level for our own components) for updated version but overall it should be quick and work fine after update without surprises.
b

Blake Romano

08/24/2022, 9:51 PM
I also noticed that the helm chart isn't up to date to v0.11 so I had to specify version in helm to v0.11 Also was wondering what Otel Collector settings I need to add to get logs injested?
s

Srikanth Chekuri

08/24/2022, 9:54 PM
There is an userguide to help you collecting and send https://signoz.io/docs/userguide/logs/. It covers some common scenarios.
The
Collecting Logs
should give you idea and the example config you should add to get it working.
View count: 2