https://signoz.io logo
Title
m

Mukesh Chaudhary

08/19/2022, 5:34 AM
Hello, I followed the instruction from docs to install signoz in kuberntes. I used different namespace (signoz) and different release name (signoz-dev). I installed the example also using this singoz example. Signoz has been installed properly. I just change values of ingress only in values.yaml file. But I am facing error
something went wrong
. The log of otel is below:
2022-08-19T04:48:22.427Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:48:32.432Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:48:42.430Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:48:52.444Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:49:02.443Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:49:12.437Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:49:22.438Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
Logs
Also in UI there is no metrics section.
p

Prashant Shahi

08/19/2022, 5:55 AM
Hi @Mukesh Chaudhary! Can you list of all pods?
kubectl -n signoz get pods
can you also share the
overwrite-values.yaml
?
m

Mukesh Chaudhary

08/19/2022, 6:10 AM
❯ kubectl -n signoz get pods NAME READY STATUS RESTARTS AGE chi-signoz-cluster-0-0-0 1/1 Running 0 24h clickhouse-operator-57959d7fd8-clmqb 2/2 Running 0 24h signoz-dev-alertmanager-0 1/1 Running 0 24h signoz-dev-frontend-69d97d7d69-lc9ht 1/1 Running 0 24h signoz-dev-otel-collector-67f955bc75-4jz2k 1/1 Running 0 24h signoz-dev-otel-collector-metrics-97b946d78-vdctk 1/1 Running 1 24h signoz-dev-query-service-0 1/1 Running 1 24h signoz-dev-zookeeper-0 1/1 Running 0 24h
I just changes ingress part only
p

Prashant Shahi

08/19/2022, 6:44 AM
@Mukesh Chaudhary That's strange.
signoz_traces
database should be created by OtelCollector's
clickhousetraces
exporter. Works fine for me.
Can you share K8s and Helm version?
Also the signoz chart version.
m

Mukesh Chaudhary

08/19/2022, 7:03 AM
I have send yaml in private message.
p

Prashant Shahi

08/19/2022, 7:29 AM
@Mukesh Chaudhary values.yaml shared by you works for me. Can you share K8s, Helm and chart versions?
m

Mukesh Chaudhary

08/19/2022, 7:31 AM
NAME            NAMESPACE       REVISION        UPDATED                                         STATUS        CHART            APP VERSION
signoz-dev      signoz          5               2022-08-18 20:10:37.387887064 +0545 +0545       deployed      signoz-0.2.5     0.10.2
p

Prashant Shahi

08/19/2022, 7:34 AM
Also share output of the following: •
kubectl version
helm version
m

Mukesh Chaudhary

08/19/2022, 7:34 AM
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: <http://version.Info|version.Info>{Major:"1", Minor:"21", GitVersion:"v1.21.6+rke2r1", GitCommit:"d921bc6d1810da51177fbd0ed61dc811c5228097", GitTreeState:"clean", BuildDate:"2021-10-28T16:49:09Z", GoVersion:"go1.16.9b7", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.24) and server (1.21) exceeds the supported minor version skew of +/-1
version.BuildInfo{Version:"v3.9.3", GitCommit:"414ff28d4029ae8c8b05d62aa06c7fe3dee2bc58", GitTreeState:"clean", GoVersion:"go1.17.13"}
p

Prashant Shahi

08/19/2022, 9:36 AM
@Mukesh Chaudhary tested with same versions of K8s and Helm.. No issues here.
It is possible that OtelCollector pod started before the clickhouse pod though there is init container to wait for it.
can you try to delete the exist otel-collector pod?
kubectl -n signoz delete pod signoz-dev-otel-collector-67f955bc75-4jz2k
m

Mukesh Chaudhary

08/20/2022, 3:05 AM
thanks it's working now but its not showing anything in dashboard:
$ export HELM_RELEASE=signoz-dev
$ export SIGNOZ_NAMESPACE=signoz
$ export HOTROD_NAMESPACE=sample-application
$ curl -sL <https://github.com/SigNoz/signoz/raw/main/sample-apps/hotrod/hotrod-install.sh|https://github.com/SigNoz/signoz/raw/main/sample-apps/hotrod/hotrod-install.sh> | bash
namespace/sample-application configured
configmap/locust-cm unchanged
configmap/scripts-cm unchanged
deployment.apps/hotrod unchanged
service/hotrod unchanged
deployment.apps/locust-master unchanged
service/locust-master unchanged
deployment.apps/locust-slave unchanged
✅ Successfully deployed HotROD sample application
p

Prashant Shahi

08/20/2022, 3:07 AM
Refer to the K8s infra metrics guide: https://signoz.io/docs/tutorial/kubernetes-infra-metrics/
m

Mukesh Chaudhary

08/20/2022, 5:02 PM
Thank you for helping me @Prashant Shahi