Hi team I m experimenting with signoz right now and trying t SigNoz Community #support

Hi team- I'm experimenting with signoz right now, ...

Scott

07/22/2024, 5:55 AM

Hi team- I'm experimenting with signoz right now, and trying to get it stood up in k8s using the helm chart. We already use clickhouse, so I was seeing if I could just attach to that installation rather than having signoz spin up a new one. After some adjustments, I've gotten many things to spin up and run correctly, including the schema migration tool. However, the query service raises an error:

Copy code

error connecting to primary db: code: 516, message: signoz: Authentication failed: password is incorrect, or there is no user with such name.

More details in 🧵 ... any help is greatly appreciated! UPDATE: fixed ... 1. quick workaround: don't use

chars in the password for credentials signoz uses to connect to the external instance. specifically the query service uses a URL format, and passwords are not properly url-encoded, so `+`'s turn into spaces according to the url. 2. https://github.com/SigNoz/charts/issues/473

Scott

07/22/2024, 5:56 AM

The schema migrations ran, and have created 3 db's with a bunch of tables. Took a little while to get the permissions just right for the new

signoz

user I created for the clickhouse instance, but all migrations ran to completion.

Scott

07/22/2024, 5:56 AM

Additionally, the init-container for query-service was able to use its credentials to connect.

Scott

07/22/2024, 5:57 AM

Finally, I spun up another clickhouse-pod in this namespace (to connect as a client using the configured host & creds), just to triple-check that there were no network restrictions or anything else about this username/password combination. Again, no issues.

Scott

07/22/2024, 5:58 AM

here are the values I am using with the signoz helm chart:

Copy code

fullnameOverride: signoz

global:
  imageRegistry: *****************

clickhouse:
  enabled: false

externalClickhouse:
  host: clickhouse-clickhouse.clickhouse-v2.svc.cluster.local
  cluster: default
  user: signoz
  existingSecret: clickhouse-auth
  existingSecretPasswordKey: password

k8s-infra:
  enabled: false

schemaMigrator:
  args:
    - --cluster-name=default

queryService:
  additionalArgs:
    - --cluster=default
  configVars:
    telemetryEnabled: false

(note the modifications to use the cluster name

default

instead of

cluster

)

Scott

07/22/2024, 6:00 AM

On the clickhouse side, there are no network restrictions for user

signoz

. Here are the grants for that user:

Copy code

- GRANT SELECT ON *.*
- GRANT CLUSTER ON *.*
- GRANT REMOTE ON *.*
- GRANT ALL ON signoz_logs.*
- GRANT ALL ON signoz_metrics.*
- GRANT ALL ON signoz_traces.*

Scott

07/22/2024, 6:03 AM

clickhouse is v24.6.1.

Scott

07/22/2024, 6:26 AM

🤯 just figured it out... my (very secure / randomly generated) password contained a

. this does not get properly url-encoded for the query service! it relies on a url like

Copy code

<tcp://host>:port?user=abc&password=$(CLICKHOUSE_PASSWORD)

and of course, any `+`'s in the password, according to any url-decoding, are turned into spaces! when i manually dropped in the password to this URL, and url encoded it myself -- e.g.

becomes

%2B

... everything works fine.

Darren Smith

07/22/2024, 7:50 AM

doh 🙂

62 Views

Open in Slack

Previous Next