I added sampling to my agent collectors to reduce ...
# supporta
Alexei Zenin
11/30/2022, 8:33 PMI added sampling to my agent collectors to reduce amount of traces collected. I placed the signozspanmetrics processor before the sampler yet the SigNoz UI seems to show the number of sampled requests vs the real number of requests as seen in the image.
Is this supposed to be the case? I thought since I would be generating metrics for 100% of the traces I would get an accurate “Number of calls”. For reference I compared to Datadog and it is 7.66K and in SigNoz it reports 766 calls, which makes sense since sampling is 10% (off by 10x).
Alexei Zenin
11/30/2022, 8:35 PMtrace pipeline config:
Copy code
traces:
receivers: [otlp]
processors: [memory_limiter, signozspanmetrics/prometheus, probabilistic_sampler, batch]
exporters: [otlphttp]Alexei Zenin
11/30/2022, 8:35 PM Copy code
probabilistic_sampler:
hash_seed: 42
sampling_percentage: 10s
Srikanth Chekuri
11/30/2022, 10:32 PMThis table is not using the spanmetrics data. Since it’s using the traces data, you don’t see the accurate total calls.
a
Alexei Zenin
11/30/2022, 10:32 PMWould it make more sense for it to display total calls and not the ones sampled?
Alexei Zenin
11/30/2022, 10:33 PMAre the other columns also not displaying span metrics data? Are P99 etc showing inaccurate numbers as well?
v
Vishal Sharma
12/01/2022, 1:48 AMKey operations table and Latency graph are fetched from traces data.
We should move them to fetching data from metrics.
cc: @Ankit Nayan
Vishal Sharma
12/01/2022, 1:59 AMCreated an issue: https://github.com/SigNoz/signoz/issues/1793
a
Alexei Zenin
01/06/2023, 4:40 PMDo you know if error rate fetched from traces or metrics? @Vishal Sharma
Alexei Zenin
01/06/2023, 4:41 PMcc @Pranay
v
Vishal Sharma
01/06/2023, 4:54 PM@Alexei Zenin Error rate is fetched from metrics
a
Alexei Zenin
01/06/2023, 4:55 PMthanks, these are the metrics computed by signozspanmetrics processor?
Alexei Zenin
01/06/2023, 6:52 PMcc @Andrew Uken
v
Vishal Sharma
01/07/2023, 2:52 AMYes
3 Views