Title
a
Alexei Zenin
11/30/2022, 8:33 PMI added sampling to my agent collectors to reduce amount of traces collected. I placed the signozspanmetrics processor before the sampler yet the SigNoz UI seems to show the number of sampled requests vs the real number of requests as seen in the image.
Is this supposed to be the case? I thought since I would be generating metrics for 100% of the traces I would get an accurate “Number of calls”. For reference I compared to Datadog and it is 7.66K and in SigNoz it reports 766 calls, which makes sense since sampling is 10% (off by 10x).
trace pipeline config:
traces:
receivers: [otlp]
processors: [memory_limiter, signozspanmetrics/prometheus, probabilistic_sampler, batch]
exporters: [otlphttp]probabilistic_sampler:
hash_seed: 42
sampling_percentage: 10s
Srikanth Chekuri
11/30/2022, 10:32 PMThis table is not using the spanmetrics data. Since it’s using the traces data, you don’t see the accurate total calls.
a
Alexei Zenin
11/30/2022, 10:32 PMWould it make more sense for it to display total calls and not the ones sampled?
Are the other columns also not displaying span metrics data? Are P99 etc showing inaccurate numbers as well?
v
Vishal Sharma
12/01/2022, 1:48 AMKey operations table and Latency graph are fetched from traces data.
We should move them to fetching data from metrics.
cc: @Ankit Nayan
Created an issue: https://github.com/SigNoz/signoz/issues/1793
a
Alexei Zenin
01/06/2023, 4:40 PMDo you know if error rate fetched from traces or metrics? @Vishal Sharma
cc @Pranay
v
Vishal Sharma
01/06/2023, 4:54 PM@Alexei Zenin Error rate is fetched from metrics
a
Alexei Zenin
01/06/2023, 4:55 PMthanks, these are the metrics computed by signozspanmetrics processor?
cc @Andrew Uken
v
Vishal Sharma
01/07/2023, 2:52 AMYes