Hello, I followed the instruction from docs to ins...
# support
m
Hello, I followed the instruction from docs to install signoz in kuberntes. I used different namespace (signoz) and different release name (signoz-dev). I installed the example also using this singoz example. Signoz has been installed properly. I just change values of ingress only in values.yaml file. But I am facing error
something went wrong
. The log of otel is below:
Copy code
2022-08-19T04:48:22.427Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:48:32.432Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:48:42.430Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:48:52.444Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:49:02.443Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:49:12.437Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
2022-08-19T04:49:22.438Z	error	clickhousetracesexporter/writer.go:101	Could not write a batch of spans	{"kind": "exporter", "name": "clickhousetraces", "error": "code: 81, message: Database signoz_traces doesn't exist"}
<http://github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/open-telemetry/opentelemetry-collector-contrib/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:101
Logs
Also in UI there is no metrics section.
p
Hi @Mukesh Chaudhary! Can you list of all pods?
kubectl -n signoz get pods
can you also share the
overwrite-values.yaml
?
m
❯ kubectl -n signoz get pods NAME READY STATUS RESTARTS AGE chi-signoz-cluster-0-0-0 1/1 Running 0 24h clickhouse-operator-57959d7fd8-clmqb 2/2 Running 0 24h signoz-dev-alertmanager-0 1/1 Running 0 24h signoz-dev-frontend-69d97d7d69-lc9ht 1/1 Running 0 24h signoz-dev-otel-collector-67f955bc75-4jz2k 1/1 Running 0 24h signoz-dev-otel-collector-metrics-97b946d78-vdctk 1/1 Running 1 24h signoz-dev-query-service-0 1/1 Running 1 24h signoz-dev-zookeeper-0 1/1 Running 0 24h
I just changes ingress part only
p
@Mukesh Chaudhary That's strange.
signoz_traces
database should be created by OtelCollector's
clickhousetraces
exporter. Works fine for me.
Can you share K8s and Helm version?
Also the signoz chart version.
m
I have send yaml in private message.
p
@Mukesh Chaudhary values.yaml shared by you works for me. Can you share K8s, Helm and chart versions?
m
Copy code
NAME            NAMESPACE       REVISION        UPDATED                                         STATUS        CHART            APP VERSION
signoz-dev      signoz          5               2022-08-18 20:10:37.387887064 +0545 +0545       deployed      signoz-0.2.5     0.10.2
p
Also share output of the following: •
kubectl version
helm version
m
Copy code
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: <http://version.Info|version.Info>{Major:"1", Minor:"21", GitVersion:"v1.21.6+rke2r1", GitCommit:"d921bc6d1810da51177fbd0ed61dc811c5228097", GitTreeState:"clean", BuildDate:"2021-10-28T16:49:09Z", GoVersion:"go1.16.9b7", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.24) and server (1.21) exceeds the supported minor version skew of +/-1
Copy code
version.BuildInfo{Version:"v3.9.3", GitCommit:"414ff28d4029ae8c8b05d62aa06c7fe3dee2bc58", GitTreeState:"clean", GoVersion:"go1.17.13"}
p
@Mukesh Chaudhary tested with same versions of K8s and Helm.. No issues here.
It is possible that OtelCollector pod started before the clickhouse pod though there is init container to wait for it.
can you try to delete the exist otel-collector pod?
Copy code
kubectl -n signoz delete pod signoz-dev-otel-collector-67f955bc75-4jz2k
m
thanks it's working now but its not showing anything in dashboard:
Copy code
$ export HELM_RELEASE=signoz-dev
$ export SIGNOZ_NAMESPACE=signoz
$ export HOTROD_NAMESPACE=sample-application
$ curl -sL <https://github.com/SigNoz/signoz/raw/main/sample-apps/hotrod/hotrod-install.sh|https://github.com/SigNoz/signoz/raw/main/sample-apps/hotrod/hotrod-install.sh> | bash
namespace/sample-application configured
configmap/locust-cm unchanged
configmap/scripts-cm unchanged
deployment.apps/hotrod unchanged
service/hotrod unchanged
deployment.apps/locust-master unchanged
service/locust-master unchanged
deployment.apps/locust-slave unchanged
✅ Successfully deployed HotROD sample application
p
Refer to the K8s infra metrics guide: https://signoz.io/docs/tutorial/kubernetes-infra-metrics/
m
Thank you for helping me @Prashant Shahi