28m Warning FailedToUpdateEndpoint endpoints/my-release-k8s-infra-otel-agent ...
a
28m Warning FailedToUpdateEndpoint endpoints/my-release-k8s-infra-otel-agent Failed to update endpoint platform/my-release-k8s-infra-otel-agent: Operation cannot be fulfilled on endpoints "my-release-k8s-infra-otel-agent": the object has been modified; please apply your changes to the latest version and try again
Hi @Srikanth Chekuri/@Ankit Nayan, any update regarding this. i am stuck with issue. i am not able to use signoz in my organisation production env
Hi @Ankit Nayan any update regarding this issue?
s
a
@Srikanth Chekuri, we removed entire namespace and its resources. after that we executed below commands. $ kubectl create namespace platform $ helm --namespace platform install my-release signoz/signoz --set frontend.image.tag="0.21.0" --set queryService.image.tag="0.21.0" --set clickhouse.persistence.size=125Gi --version 0.17.0
Copy code
helm --namespace platform install my-release signoz/signoz --set frontend.image.tag="0.21.0" --set queryService.image.tag="0.21.0" --set clickhouse.persistence.size=125Gi --version 0.17.0
coalesce.go:175: warning: skipped value for zookeeper.initContainers: Not a table.
NAME: my-release
LAST DEPLOYED: Thu May 2 10:33:07 2024
NAMESPACE: platform
STATUS: deployed
REVISION: 1
NOTES:
1. You have just deployed SigNoz cluster:
• frontend version: '0.21.0' • query-service version: '0.21.0' • alertmanager version: '0.23.0-0.2' • otel-collector version: '0.79.1' • otel-collector-metrics version: '0.79.1' 2. Get the application URL by running these commands: export POD_NAME=$(kubectl get pods --namespace platform -l "app.kubernetes.io/name=signoz,app.kubernetes.io/instance=my-release,app.kubernetes.io/component=frontend" -o jsonpath="{.items[0].metadata.name}") echo "Visit http://127.0.0.1:3301 to use your application" kubectl --namespace platform port-forward $POD_NAME 3301:3301 If you have any ideas, questions, or any feedback, please share on our Github Discussions: https://github.com/SigNoz/signoz/discussions/713 Improving helm chart for SigNoz · SigNoz signoz · Discussion #713 github.com
We did the changes to the yaml file in link you provided . even though same issue persist
kindly check in the screenshot
k8s version is 1.23.17
kindly help me in this regard
s
We did the changes to the yaml file in link you provided
What changes did you do?
kindly check in the screenshot
This doesn't help at all. Please check why your clickhouse as init errors.
a
@Goutham Sridhar can you reply to this
g
@Srikanth Chekuri We get below errors in logs and describing pod respectively. Even the kubernetes events shows only this error message.
Copy code
Error from server (BadRequest): container "clickhouse" in pod "chi-my-release-clickhouse-cluster-0-0-0" is waiting to start: PodInitializing
Copy code
State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
s
Please share more information. What are the logs when the clickhouse exited?
a
@Kishore MC, please provide the necessary info
k
@Srikanth Chekuri these are the only logs we are getting . and also i have sent you events logs as well
a
Hi @Srikanth Chekuri, any update on this ? we have been struck for long time
@Srikanth Chekuri, kindly help us on this. In the production signoz is not working.
@Ankit Nayan and @Srikanth Chekuri we are unable to install the signoz 21 version in our production environment. kindly help us in this regard
s
As I mentioned elsewhere, the 0.21 version is very old and we can't help with such an old version. Version aside, the errors mentioned ("the object has been modified; please apply your changes to the latest version and try again", CrashLoopBackOff of clickhouse pod etc..) are not signoz specific errors but rather generic k8s management.
k
Hi Team, We upgraded our K8s cluster to 1.27 and we noticed that these pods are not coming up. This looks similar to the above issue we reported previously. We assume that there is some problem with signoz 0.21 version. Please look into it ASAP. Attached screenshot for your reference
a
@Srikanth Chekuri and @Ankit Nayan, when we upgraded K8s version, the signoz 21 version is not at all supporting. only signoz 23 version supporting in k8s version more than 1.23. can you kindly justify my statement?
k
@Srikanth Chekuri and @Ankit Nayan, Could you please provide us your valuable inputs on the above statement so that we can proceed with signoz v0.23 upgrade
s
Hi, It's very old and we can't reliably tell it's compatibility with k8s version. If it works with some new version, please use it.
k
Sure, Thanks @Srikanth Chekuri
142 Views