https://signoz.io logo
Join Slack
Powered by
Hey all! I am looking at diving into our company u...
# support
b
Hey all! I am looking at diving into our company using Signoz. I want to have a POC where we could have one Signoz instance that can recieve data from multiple data sources from different clusters. However I want to ensure that the transfer of this data is secured ideally by some sort of API key, I tried reading the docs but I didn't see anything. Does anyone have any experience with this?
v
There are some documentations over here https://github.com/open-telemetry/opentelemetry-collector/tree/main/config/configauth However I didn't managed to make it work on my side. If you find a way tell us 😛
b
This looks like exactly what I was looking for thank you!
p
@Blake Romano @Valentin Baert There is a detailed guide here that should be useful: https://medium.com/opentelemetry/securing-your-opentelemetry-collector-1a4f9fa5bd6f
v
Yes I saw that but in my case I'm using the otel operator for kubernetes and with the sidecar deployment mode I don't know how to configure tls certificates
p
You could add additional host that you will be using to communicate between OtelCollectors in CSR JSON and generate the certificates locally.
Followed by 
create secret tls
with certificate and key. You can refer here: https://aws-otel.github.io/docs/getting-started/operator#install-the-opentelemetry-operator-to-your-eks-cluster
b
I super appreciate the tips @Prashant Shahi this is def something I will need if I want to open the collector on the public internet to allow for cross cluster/vpc communication
p
yeah, that definitely makes sense when you have cross cluster/vpc communication. We will be thoroughly be testing and creating tutorials on the same in some time.
Meanwhile, if you need a simpler solution, you could use BasicAuth as we have tested it as well. Config would look something like this:
Copy code
otlp:
    protocols:
      http:
        auth:
          authenticator: basicauth
        cors:
          allowed_origins:
            - <http://localhost>:*
...
extensions:
  health_check: {}
  zpages: {}
  basicauth:
    htpasswd: 
      inline: |
        ${BASIC_AUTH_USERNAME}:${BASIC_AUTH_PASSWORD}

service:
  extensions: [health_check, zpages, basicauth]
while, you would have to include those environment variables for OTel collector:
Copy code
environment:
      - BASIC_AUTH_USERNAME=signoz
      - BASIC_AUTH_PASSWORD=password
b
and this authentication would work for all the Open Telemetry recievers like Prometheus etc;?
p
This would work for OTLP. So, I reckon you would need to use another OtelCollector agent in between.
metrics/traces in cluster A --> Otel in cluster A ---- across cluster/vpc ----> Otel in cluster B (signoz otel)
b
Ahh okay. I am still making trying to wrap my head around the Open Telemetry concepts with Signoz but that makes sense. As long as I know it’s possible with some basic ideas as to where I need to bring it that is super helpful
One other quick question @Prashant Shahi what is the difference between Signoz Alertmanager and Prometheus Alert Manager? We already have Prom Alert Manager setup. Could we configure that with Signoz to integrate the two? Reading the signoz/alertmanager readme it seems to be copied from Prom Alert Manager
p
@Blake Romano yes, Signoz Alertmanager is a fork used for supporting alerts in SigNoz. any particular reason that you would want to integrate them? I am not sure how that would work though. @Amol Umbark would be the best person to answer that.
b
Well we already have a lot of setup with the Prometheus Rules CRD and also we would like to be able to programatically create alerts for services using source code
a
@Blake Romano the signoz's alertmanger just creates a few apis to add channels dynamically from the SigNoz's UI. Also alertmanager initialization is not dependent on config.yaml anymore. But we have not tested signoz with prometheus alertmanager and thus can break.
also can you please explain this in somewhat more detail?
we would like to be able to programatically create alerts for services using source code
Are you talking about applications in python/nodejs/go creating alerts? Or just maintaining a list of yamls containing alerts and rules?
b
Be able to maintain a list of yamls that have the rules and alerts more specifically ^
👍 1
is that doable?
a
It won't be doable right now but I think it can be built. We will be releasing a better Ui for writing queries for alerts and dashboard with autosugestions on metric names and labels/tags. So, our data format for rules and alerts is going to change but we can build a feature where we initialise the rules from yamls.
Can you raise a feature request for this at out github? Would be useful to track it publically
b
yup can do. I think for our use case using a CRD and deploying it that way would be helpful
a
I would suggest give signoz a deeper try and if this the only thing restricting you to move to prod. We shall pick it up in our sprints.
b
I added it. I am hoping to have a POC in about a month which is when this feature would be more crucial in our steps towards transition
a
Cool 👍
b
@Prashant Shahi is there a way to have an ingress for the Otel collector? I’d like to open up the authenticated port for the Signoz Otel Collector on an ingress? Is this something I could PR the helm chart?
p
we haven't added ingress yet for otel collector and otel collector metrics. It could be great if you could raise a PR with those.
b
I can probably do that. I have a PR out on helm chart now regarding a bug if you don’t mind checking it out 🙂
p
Thanks for the PR! I have reviewed and left comment.
✅ 1
52 Views
Open in Slack
PreviousNext
Powered by