Does anybody know which tables these metrics are s...
# support
a
Does anybody know which tables these metrics are stored in? https://github.com/SigNoz/signoz-otel-collector/tree/main/processor/signozspanmetricsprocessor. I am trying to query them outside of SigNoz and create some alerts based on error rates, http latency etc. Any example queries that I could use to do these in Clickhouse SQL?
p
@Alexei Zenin Curious, why do you want to query them outside SigNoz and create alerts? What was not supported in the SigNoz alerting capabilities?
a
@Alexei Zenin querying metrics from clickhouse without our query interface might be difficult because of query complexity. Can you share what you are trying to achieve?
tables ->
signoz_metrics.time_series_v2
and
signoz_metrics.samples_v2
a
Our setup is to have alerts in Grafana since all of them will be there for alerting on all datasources (cloudwatch, prometheus, clickhouse and any future ones that come up). I’ve played around with the dashboard viewer and tried out plotting average latency for a service but am getting negative numbers as seen below:
hmm, but that PR seems more to be focused on the service page
would upgrading to 0.13 fix this in the dashboard page?
p
@Srikanth Chekuri do you have ideas on this?
a
@Alexei Zenin you are choosing incorrect
metric_name
. There are 8 steps to the tutorial using a formula
A/B
. The 2 metrics are
signoz_latency_sum
and
signoz_latency_count
. Make sure you have followed all the steps correctly
a
ah right, did it fully this time, seems to be the same issue:
a
@Srikanth Chekuri will be able to dig deeper 🙂
s
Use the legend {{service_name}} in all sections. That aside, Can you share the graphs for A, B, C separately with other two disabled in each?
a
Only B:
Only A:
Signoz version 0.11.2, collector version signoz/signoz-otel-collector:0.55.3
s
The chart displays the negatives when the counters reset. Was there any incident of otel-collector-metrics restarting?
a
we have our collectors restart often with ec2 spot instances
s
That resets the counters, and the frequent resets lead to this graph where the value becomes negative. But otherwise if the y-axis is selected with appropriate scale (millis in this instance I believe) do you see the correct value for positive numbers?
a
Hello Signoz Team, I am trying to understand signoz metrics table in clickhouse. I found two tables, time_series_v2 and time_series_v3. I found detailed information about the v2 table only. Can anyone explain to me why V3 table is required?  My current signoz version is v0.33.0
147 Views