:wave: Hello, Team! Is it possible to setup signoz...
# support
a
👋 Hello, Team! Is it possible to setup signoz with istio. So It can use data /metrics generate by the istio mesh targeting k8s services ? I stumbled upon an open issue but unable to understand how to get this up and running . Any help or guidance is much appreciated .
l
Hi Aman, which part are you trying to achieve? I'm using Istio and SigNoz
Do you want Kiali graphs be rendered by SigNoz?
a
I have my application developed with opentelemetry and istio is running on the cluster. So istio should aggregate the data and send it to kiali and then signoz will use that data to visualise
p
@Prashant Shahi would you have more insights on this
p
@Aman Upadhyay You can enable Zipkin receiver in SigNoz OtelCollector, and send traces from Istio. • Include the following in OtelCollector config:
Copy code
receivers:
  zipkin:
    endpoint: 0.0.0.0:9411
  ...
• Include
zipkin
in the
traces
pipeline.
Copy code
pipelines:
    traces:
      receivers: [zipkin, ...]
      ...
• Configure SigNoz OtelCollector Zipkin address in Istio https://istio.io/latest/docs/ops/integrations/zipkin/#option-2-customizable-install
For metrics, you can include the scrape config from Istio guide to either SigNoz OtelCollectorMetrics or OtelDeployment (under k8s-infra chart). https://istio.io/latest/docs/ops/integrations/prometheus/#option-2-customized-scraping-configurations
cc @Aman Upadhyay @Lucas Carlos
l
Hey @Prashant Shahi yesterday @Srikanth Chekuri and I found a memory leak on query service generated by istio metrics
query-service hit 13gb and never dropped
p
Hi Lucas! Thanks for mentioning it on the thread.
could it be caused by frequent metrics collection i.e. scrape interval set to small duration?
Or, was it something else?
l
not actually, 10k samples broke the whole thing
what's different is that each metric has ~40 labels
p
oh, I see.
we could test out with minimal labels using
labeldrop
or`labelkeep` , to see if the issue persists with Istio metrics when labels are reduced.
s
It’s probably specific to their environment. I think I misquoted what we were doing.
a
@Pranay @Prashant Shahi @Lucas Carlos Thanks for responding to my query. Will follow the approach suggest by you guys .
l
Hey @Aman Upadhyay the scraping works fine, we just found that the dashboards must be edited, just remove
max(up *)
statement from the queries, this will try to load all
up
within your queries and potentially overload query-service
a
Cool
p
@Lucas Carlos Whenever you find time, would be great if you can share a note on how you monitor Istio with SigNoz. We can also publish in tutorials sections of our docs if that works for you. Would be helpful for other community members who are looking to monitor Istio 🙂
l
Yes, for sure
p
Awesome! If it's simpler - you can just open a PR here - https://github.com/SigNoz/signoz.io/tree/main/docs/tutorial