How the 99.9th percentile can be calculated I can't find any refence document

No, there is no support for the 99.9th percentile. Only P99 in supported at the moment in query builder. However, you can use promql expression with 99.9 for p99.9

I have a similar problem regarding percentile calculation for a metrics value in a dashboard. I found there is no percentile aggregation at all for metrics. So I tried to use a PromQL expression but it always shows "No Data". Did I do the query right for a metric called "lcp"?

histogram_quantile(0.75, sum(rate(lcp[5m])))

Btw. I'm new to PromQL and just guessed the syntax from an example in the Prometheus docs.

The metric type must be histogram; otherwise, the percentile option is not shown in the query builder. Based on metric name it is unlikely the metric is histogram type.

So it is not possible to get a percentile value for a "plain" metric?

I am not sure I follow. Each metric has some type associated with it; either gauge, count, histogram, summary etc.. depending on the metric type, certain aggregation options are shown. It doesn't make sense to try to get a Pxx value out of a counter metric. Only histogram metrics produce meaningful percentile values. What are you trying to achieve?

We want to compute the core web vitals values and do visualizations/alerting. We want to get close to the recommendations, e.g. https://web.dev/articles/inp, where the 75. percentile is used to get a meaningful value.

clearly, this is talking about duration which means the right way to measure duration is to use the histogram. i don't understand what you meant by plain metric? Did you mean raw measurements?

With plain, I mean just a time series of plain values. Currently we do collect all measures as gauge metrics. If we collect the duration based values of the core web vitals (LCP and INP) as histograms we can use quantiles. But there is also the CLS value which measures the visual page stability. So this is not a duration value. But here it is adviced to use the 75. percentile too. So we would be forced to use histogram type too. Right?

I didn't review the web vitals doc, so I am not sure why it's gauge. If you are interested in duration measurements, they should be histograms. The gauge type is not appropriate for percentile because • they have a last value aggregation i.e only last observed value during the interval is reproted which is not a representative metric.