Hi Getting below error on v0 15 0 I am not able to view any SigNoz Community #support

Hi, Getting below error on v0.15.0. I am not able...

Shreyash

02/03/2023, 9:00 AM

Hi, Getting below error on v0.15.0. I am not able to view any traces on the dashboard

Copy code

"numberTagMap":{"process.pid":68,"thread.id":11890}}}], "error": "clickhouse [Append]:  clickhouse: expected 32 arguments, got 35"}
<http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).writeBatch|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).writeBatch>
	/src/exporter/clickhousetracesexporter/writer.go:137
<http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:109
2023-02-03T08:01:54.178Z	error	clickhousetracesexporter/writer.go:110	Could not write a batch of spans	{"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "error": "clickhouse [Append]:  clickhouse: expected 32 arguments, got 35"}
<http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:110

Pranay

02/03/2023, 9:25 AM

@Shreyash Which version of SigNoz did you upgrade from?

Vishal Sharma

02/03/2023, 9:28 AM

@Shreyash How did you upgrade? Please let us know steps. Your collector doesn’t seems to be upgraded correctly.

Shreyash

02/03/2023, 9:31 AM

https://signoz.io/docs/operate/kubernetes/#upgrade-signoz-cluster Followed these steps: 1-2

Shreyash

02/03/2023, 9:35 AM

@Pranay 0.13

Vishal Sharma

02/03/2023, 9:35 AM

@Shreyash Can you please send result of this:

helm search repo signoz --versions

Shreyash

02/03/2023, 9:36 AM

sure

Shreyash

02/03/2023, 9:36 AM

Copy code

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/shreyash.ramtekkar/.kube/config
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/shreyash.ramtekkar/.kube/config
NAME               	CHART VERSION	APP VERSION	DESCRIPTION                                      
signoz/signoz      	0.10.2       	0.15.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.10.1       	0.15.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.10.0       	0.15.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.9.1        	0.14.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.9.0        	0.14.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.8.1        	0.13.1     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.8.0        	0.13.1     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.7.1        	0.13.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.7.0        	0.13.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.6.0        	0.12.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.5.1        	0.11.4     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.5.0        	0.11.3     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.7        	0.11.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.6        	0.11.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.5        	0.11.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.4        	0.11.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.3        	0.11.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.2        	0.11.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.1        	0.11.1     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.4.0        	0.11.1     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.3.3        	0.11.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.3.2        	0.11.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.3.1        	0.11.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.3.0        	0.10.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.2.5        	0.10.2     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.2.4        	0.10.1     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.2.3        	0.10.1     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.2.2        	0.10.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.2.1        	0.10.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.2.0        	0.10.0     	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.1.4        	0.9.2      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.1.3        	0.9.2      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.1.2        	0.9.2      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.1.1        	0.9.1      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.1.0        	0.9.0      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.20       	0.8.2      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.19       	0.8.2      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.18       	0.8.2      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.17       	0.8.1      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.16       	0.8.0      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.15       	0.8.0      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.14       	0.8.0      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.13       	0.8.0      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.12       	0.7.5      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.11       	0.7.4      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.10       	0.7.1      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.9        	0.7.1      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.8        	0.6.2      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.7        	0.6.1      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.6        	0.6.1      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.5        	0.6.0      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.4        	0.5.4      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.3        	0.5.4      	SigNoz Observability Platform Helm Chart         
signoz/signoz      	0.0.2        	0.5.4      	SigNoz Observability Platform Helm Chart         
signoz/alertmanager	0.5.2        	0.5.0      	The Alertmanager handles alerts for SigNoz.      
signoz/alertmanager	0.5.1        	0.5.0      	The Alertmanager handles alerts for SigNoz.      
signoz/alertmanager	0.5.0        	0.5.0      	The Alertmanager handles alerts for SigNoz.      
signoz/clickhouse  	23.8.2       	22.8.8     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.8.1       	22.8.8     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.8.0       	22.8.8     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.7.0       	22.8.8     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.6.0       	22.8.8     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.5.4       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.5.3       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.5.2       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.5.1       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.5.0       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.4.0       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.3.3       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.3.2       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.3.1       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	23.3.0       	22.4.5     	A Helm chart for ClickHouse                      
signoz/clickhouse  	16.0.6       	21.12.3.32 	A Helm chart for ClickHouse                      
signoz/clickhouse  	16.0.5       	21.12.3.32 	A Helm chart for ClickHouse                      
signoz/clickhouse  	16.0.4       	21.12.3.32 	A Helm chart for ClickHouse                      
signoz/clickhouse  	16.0.3       	21.12.3.32 	A Helm chart for ClickHouse                      
signoz/clickhouse  	16.0.2       	21.12.3.32 	A Helm chart for ClickHouse                      
signoz/clickhouse  	16.0.1       	21.12.3.32 	A Helm chart for ClickHouse                      
signoz/clickhouse  	16.0.0       	21.12.3.32 	A Helm chart for ClickHouse                      
signoz/clickhouse  	9.1.0        	21.7       	A Helm chart for ClickHouse                      
signoz/k8s-infra   	0.6.3        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.6.2        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.6.1        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.6.0        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.5.0        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.4.2        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.4.1        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.4.0        	0.66.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.3.1        	0.63.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.3.0        	0.63.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.2.2        	0.55.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.2.1        	0.55.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.2.0        	0.55.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.1.4        	0.55.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.1.3        	0.55.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.1.2        	0.55.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.1.1        	0.55.0     	Helm chart for collecting metrics and logs in K8s
signoz/k8s-infra   	0.1.0        	0.55.0     	Helm chart for collecting metrics and logs in K8s

Vishal Sharma

02/03/2023, 9:46 AM

Can you please please restart otel collector and share initial logs (first 100 lines should be enough)

Prashant Shahi

02/03/2023, 10:42 AM

@Shreyash It's possible that you are using older version of SigNoz OtelCollector. You need to make sure to sync your custom

override-values.yaml

, if you have included older OtelCollector version in it.

Prashant Shahi

02/03/2023, 10:43 AM

https://github.com/SigNoz/charts/blob/main/charts/signoz/values.yaml#L992-L997 https://github.com/SigNoz/charts/blob/main/charts/signoz/values.yaml#L1492-L1497

Prashant Shahi

02/03/2023, 10:50 AM

@Shreyash You can check the otel-collector versions using the following:

Copy code

kubectl -n platform get deployment -o wide | grep signoz-otel-collector

Shreyash

02/07/2023, 12:05 PM

hii @Prashant Shahi, Sorry for the late response

Copy code

kubectl -n platform get deployment -o wide | grep signoz-otel-collector
signoz-release-otel-collector              1/1     1            1           67d   signoz-release-otel-collector                                                   <http://docker.io/signoz/signoz-otel-collector:0.66.3|docker.io/signoz/signoz-otel-collector:0.66.3>                                              <http://app.kubernetes.io/component=otel-collector,app.kubernetes.io/instance=signoz-release,app.kubernetes.io/name=signoz|app.kubernetes.io/component=otel-collector,app.kubernetes.io/instance=signoz-release,app.kubernetes.io/name=signoz>
signoz-release-otel-collector-metrics      1/1     1            1           67d   signoz-release-otel-collector-metrics                                           <http://docker.io/signoz/signoz-otel-collector:0.66.3|docker.io/signoz/signoz-otel-collector:0.66.3>                                              <http://app.kubernetes.io/component=otel-collector-metrics,app.kubernetes.io/instance=signoz-release,app.kubernetes.io/name=signoz|app.kubernetes.io/component=otel-collector-metrics,app.kubernetes.io/instance=signoz-release,app.kubernetes.io/name=signoz>

Prashant Shahi

02/07/2023, 12:14 PM

No problem. It looks good. You have latest version of SigNoz otel-collector.

Prashant Shahi

02/07/2023, 12:15 PM

@Shreyash can you restart the pods and share new container logs? You can delete those pods to spawn new ones.

Shreyash

02/07/2023, 12:16 PM

Copy code

thread.name:http-nio-8085-exec-4]"}, "error": "clickhouse [Append]:  clickhouse: expected 32 arguments, got 35"}
<http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).writeIndexBatch|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).writeIndexBatch>
	/src/exporter/clickhousetracesexporter/writer.go:201
<http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).writeBatch|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).writeBatch>
	/src/exporter/clickhousetracesexporter/writer.go:135
<http://github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter|github.com/SigNoz/signoz-otel-collector/exporter/clickhousetracesexporter.(*SpanWriter).backgroundWriter>
	/src/exporter/clickhousetracesexporter/writer.go:109
2023-02-07T12:08:47.296Z	error	clickhousetracesexporter/writer.go:137	Could not write a batch of spans to index table: 	{"kind": "exporter", "data_type": "traces", "name": "clickhousetraces", "batch": [

this is the only error i am getting in the pod logs

Prashant Shahi

02/07/2023, 2:02 PM

From what I recall, this issue happens when there is version mismatch between query-service and signoz-otel-collector. But this somehow doesn't seem be the case here. @Srikanth Chekuri @Vishal Sharma Can you guys look into this?

Vishal Sharma

02/07/2023, 2:12 PM

@Prashant Shahi Migration might have failed here. @Shreyash Can you please please restart otel collector and share initial logs (first 100 lines should be enough)?

Prashant Shahi

02/07/2023, 3:05 PM

@Vishal Sharma it was already done, check above for logs.

3 Views

Open in Slack

Previous Next