https://signoz.io
Join Slack
Hello, I have observed a significant increase in CPU utilization after the Signoz version upgrade to...
s

saurabh biramwar

over 1 year ago
Hello, I have observed a significant increase in CPU utilization after the Signoz version upgrade to v0.41.1. Additionally, we're encountering the following error in the ClickHouse pod. 2024.03.21 130442.644576 [ 1011 ] {bc235080-bd8e-4a26-99d6-69e10ec9bee9} <Error> TCPHandler: Code: 210. DB:NetException I/O error: Broken pipe, while writing to socket ([:ffff10.0.145.23]:9000 -> [:ffff10.0.129.149]:33502). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. DB:Exception:Exception(DB:Exception:MessageMasked&&, int, bool) @ 0x000000000c800f1b in /usr/bin/clickhouse 1. DB:NetException:NetException<String, String, String>(int, FormatStringHelperImpl<std:type identity&lt;String&gt;:type, std:type identity&lt;String&gt;:type, std:type identity&lt;String&gt;:type>, String&&, String&&, String&&) @ 0x000000000caa69a1 in /usr/bin/clickhouse 2. DB:WriteBufferFromPocoSocket:nextImpl() @ 0x000000000caa733e in /usr/bin/clickhouse 3. DB:TCPHandler:runImpl() @ 0x000000001292120f in /usr/bin/clickhouse 4. DB:TCPHandler:run() @ 0x0000000012933eb9 in /usr/bin/clickhouse 5. Poco:NetTCPServerConnection:start() @ 0x00000000153a5a72 in /usr/bin/clickhouse 6. Poco:NetTCPServerDispatcher:run() @ 0x00000000153a6871 in /usr/bin/clickhouse 7. Poco:PooledThread:run() @ 0x000000001549f047 in /usr/bin/clickhouse 8. Poco:ThreadImpl:runnableEntry(void*) @ 0x000000001549d67d in /usr/bin/clickhouse
s
r
+2
  • 4
  • 7
  • 131
This message was deleted.
s

Slackbot

almost 2 years ago
This message was deleted.
s
m
+3
  • 5
  • 38
  • 131
This message was deleted.
s

Slackbot

over 2 years ago
This message was deleted.
s
n
+2
  • 4
  • 16
  • 131
Hi Team, I am trying to do a fresh install of signoz (self-hosted) using helm chart version 0.57.0 (...
k

kankan ghosh

11 months ago
Hi Team, I am trying to do a fresh install of signoz (self-hosted) using helm chart version 0.57.0 (signoz appVersion 0.59.0). but schema-migrator sync and async init pods are failing with the same error as below. Any idea what this error means? also, another question that I have - why is schema-migrator needed for a fresh setup of signoz? Error message: 
{"L":"info","timestamp":"2024-12-12T07:36:09.723Z","C":"signozschemamigrator/main.go:162","M":"Migrating up"}
{"L":"info","timestamp":"2024-12-12T07:36:09.723Z","C":"schema_migrator/manager.go:589","M":"Running migrations up sync"}
{"L":"info","timestamp":"2024-12-12T07:36:09.723Z","C":"schema_migrator/manager.go:551","M":"Checking if migration should run","db":"signoz_metrics","migration_id":1000,"versions":[]}
{"L":"info","timestamp":"2024-12-12T07:36:09.723Z","C":"schema_migrator/manager.go:568","M":"Fetching migration status","query":"SELECT * FROM signoz_metrics.schema_migrations_v2 WHERE migration_id = 1000 SETTINGS final = 1;"}
*{"L":"error","timestamp":"2024-12-12T07:36:09.724Z","C":"schema_migrator/manager.go:576","M":"Failed to fetch migration status","error":"code: 115, message: Unknown setting final","S":"<http://github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).shouldRunMigration|github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).shouldRunMigration>*\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator/manager.go:576\<http://ngithub.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).MigrateUpSync|ngithub.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).MigrateUpSync>\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator/manager.go:617\nmain.registerSyncMigrate.func1\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/main.go:163\<http://ngithub.com/spf13/cobra.(*Command).execute|ngithub.com/spf13/cobra.(*Command).execute>\n\t/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:985\<http://ngithub.com/spf13/cobra.(*Command).ExecuteC|ngithub.com/spf13/cobra.(*Command).ExecuteC>\n\t/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117\<http://ngithub.com/spf13/cobra.(*Command).Execute|ngithub.com/spf13/cobra.(*Command).Execute>\n\t/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041\nmain.main\n\t/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/main.go:70\nruntime.main\n\t/opt/hostedtoolcache/go/1.23.3/x64/src/runtime/proc.go:272"}
panic: code: 115, message: Unknown setting final
 
goroutine 1 [running]:
<http://github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).shouldRunMigration(0xc0000e7520|github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).shouldRunMigration(0xc0000e7520>, {0xcb5fa1, 0xe}, 0x3e8, {0x13b6a40, 0x0, 0x0})
/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator/manager.go:577 +0xb85
<http://github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).MigrateUpSync(0xc0000e7520|github.com/SigNoz/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator.(*MigrationManager).MigrateUpSync(0xc0000e7520>, {0xe21c60, 0x13b6a40}, {0x13b6a40, 0x0, 0x0})
/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/schema_migrator/manager.go:617 +0x27a
main.registerSyncMigrate.func1(0xc0001ca608, {0xcb1c0b?, 0x4?, 0xcb1c0f?})
/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/main.go:163 +0xfe9
<http://github.com/spf13/cobra.(*Command).execute(0xc0001ca608|github.com/spf13/cobra.(*Command).execute(0xc0001ca608>, {0xc0001ce8a0, 0x3, 0x3})
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:985 +0xaaa
<http://github.com/spf13/cobra.(*Command).ExecuteC(0xc0001ca308)|github.com/spf13/cobra.(*Command).ExecuteC(0xc0001ca308)>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff
<http://github.com/spf13/cobra.(*Command).Execute(...)|github.com/spf13/cobra.(*Command).Execute(...)>
/home/runner/go/pkg/mod/github.com/spf13/cobra@v1.8.1/command.go:1041
main.main()
/home/runner/work/signoz-otel-collector/signoz-otel-collector/cmd/signozschemamigrator/main.go:70 +0x1f9
k
n
  • 2
  • 5
  • 130
Hello, I'm trying to work on the Alertmanager for this issue: <https://github.com/SigNoz/signoz/issu...
i

Ileo

about 1 year ago
Hello, I'm trying to work on the Alertmanager for this issue: https://github.com/SigNoz/signoz/issues/5095. I installed Go version 1.22.5 (linux/amd64), but when I execute "make," I get an error related to the Go version.
i
n
s
  • 3
  • 3
  • 130
Hello Team, Could you please assist me with an issue I'm facing? I have installed and configured Sig...
p

Prathap ch

about 1 year ago
Hello Team, Could you please assist me with an issue I'm facing? I have installed and configured SigNoz on an AWS EKS cluster. It is working as expected with the OTEL collector endpoint pointing to the default service/pod endpoint like this: 
otlp:
        endpoint: <http://signoz-otel-collector.platform.svc.cluster.local:4317>
Our team has requested to externalize this endpoint using Ingress, so we can use the same endpoint across different clusters instead of configuring OTEL in each cluster. We have installed the AWS Load Balancer Controller to use as the ingress controller. I created an Ingress resource using the following specification referenced in the Helm values file: 
otelCollector:
  ingress:
    # -- Enable ingress for otelCollector
    enabled: true
    # -- Annotations to otelCollector Ingress
    annotations:
      alb.ingress.kubernetes.io/load-balancer-name: signoz-collector-lb
      kubernetes.io/ingress.class: alb
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/listen-ports: '[{"HTTP":80}, {"HTTPS":443}]'
      alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:XXXXXXXXX:certificate/XXXXXXXXXXX
      alb.ingress.kubernetes.io/ip-address-type: ipv4
      alb.ingress.kubernetes.io/target-type: ip
    # -- Frontend Ingress Host names with their path details
    hosts:
      - host: <HOST_URL>
        paths:
          - path: /
            pathType: ImplementationSpecific
            port: 4318
I was able to create the Load Balancer, but it is failing the health check, which prevents access to the endpoint from outside. Can someone please check and help me resolve this issue? I am using EKS version 1.27 and the Helm chart version is "signoz-0.44.0".
p
n
+2
  • 4
  • 7
  • 129
Hello, Does anyone have metrics on CPU requirements based on the number of requests per second? Our...
t

Trent Lawson

about 1 year ago
Hello, Does anyone have metrics on CPU requirements based on the number of requests per second? Our installation has 8 cores and is receiving around 150 requests per second, the CPU sits around 100% and we are dropping a number of requests. https://www.restack.io/docs/signoz-knowledge-signoz-hardware-requirements 
CPU: The minimal CPU requirement is 4 cores. To ensure smoother operation and to accommodate higher workloads, 8 cores are recommended.
850979 101       20   0   44.0g   1.7g  62076 S 471.2  17.7  29208:00 clickhouse-serv
1592062 root      20   0 7168288   4.2g  93568 S 301.7  44.0   3:18.29 signoz-collecto
Any details on upper limits would be helpful.
t
n
p
  • 3
  • 6
  • 129
Hello, I have added a custom resource attribute to my traces via the sidecar collector: ``` re...
e

Eric Thompson

over 1 year ago
Hello, I have added a custom resource attribute to my traces via the sidecar collector: 
resource/k8s:
        attributes:
          - key: test.foo
            value: bar
            action: insert
          - key: k8s.cluster.name
            from_attribute: k8s-cluster
            action: insert
          - key: k8s.namespace.name
            from_attribute: k8s-namespace
            action: insert
I can use that resource attribute to filter in the Traces area of Signoz, but the Services area will not show it as a filter option - only the signoz collector id shows up as filterable. Any ideas on why it wouldn't be allowing the Service filter on another but the collector id even thought the data is there and filterable in the traces area?
e
p
c
  • 3
  • 6
  • 129
I'm trying to scrape application pod (prometheus) metrics using the k8s-infra chart. I've not been a...
j

Jens Kristian Geyti

over 1 year ago
I'm trying to scrape application pod (prometheus) metrics using the k8s-infra chart. I've not been able to find much documenation, but I note that the following configuration exists https://github.com/SigNoz/charts/blob/main/charts/k8s-infra/values.yaml#L349-L352 , which is briefly being discussed at https://community-chat.signoz.io/t/8493962/hi-team-i-ve-installed-signoz-via-helm-chart-initially-there. Setting those labels on my pod exposing a prometheus page doesn't seem to get picked up though. Has anyone successfully scraped prometheus metrics with the setup in the k8s-infra chart?
j
s
  • 2
  • 25
  • 128
Hey Guys, I am trying to send alert:rotating_light: notification over Email, I am getting some error...
s

Siddhant Gupta

over 1 year ago
Hey Guys, I am trying to send alert🚨 notification over Email, I am getting some error of channel not being configured properly to send email for even a test alert. And I am not getting any mails even for an obvious conditions for alert. 
Failed to send a test message to this channel, please confirm that the parameters are set correctly
This was the error message popped up when trying to set email channel and send a test mail Can you send me the procedure of setting mail based alerts, or does it require any prior SMTP configuration. Thanks and regards
s
r
+2
  • 4
  • 5
  • 128
Previous161718Next

SigNoz Community

SigNoz is an open-source APM. It helps developers monitor their applications & troubleshoot problems, an open-source alternative to DataDog, NewRelic, etc.

Powered by