An average can hold steady while a small share of requests gets dramatically slower. A latency histogram helps locate that tail. With a classic Prometheus histogram, each bucket is a counter for requests whose duration was at most the bucket boundary.
histogram_quantile(
0.95,
sum by (le, route) (
rate(http_request_duration_seconds_bucket[5m])
)
)
Keep the le label when aggregating classic buckets. Summing the bucket rates across replicas lets Prometheus estimate the percentile of their combined observations. Averaging a p95 calculated for each replica gives each replica’s percentile a vote without accounting for how much traffic it handled.
Put request volume beside the percentile
During a quiet window, a few requests can move the estimated p95 sharply. Show the request count or rate next to the percentile. When a latency target has a fixed threshold, consider showing the fraction of requests in the bucket at that threshold too.
Bucket boundaries matter. If most values land in one wide bucket near your target, the estimate can look more precise than the measurement. Place extra boundaries around the latencies you need to distinguish, and use a bounded set of route names so every user ID or raw URL does not become a new time series.