Simple & Optimal Quantile Sketch: Combining Greenwald-Khanna with Khanna-Greenwald

Elena Gribelyuk, Pachara Sawettamalya, Hongxun Wu, Huacheng Yu

Abstract:

Estimating the $\epsilon$-approximate quantiles or ranks of a stream is a fundamental task in data monitoring. Given a stream $x_1,..., x_n$ from a universe $\mathcal{U}$ with total order, an additive-error quantile sketch $\mathcal{M}$ allows us to approximate the rank of any query $ y\in \mathcal{U}$ up to additive $\epsilon n$ error.

In 2001, Greenwald and Khanna gave a deterministic algorithm (GK sketch) that solves the $\epsilon$-approximate quantiles estimation problem using $O(\epsilon^{-1} \log(\epsilon n))$ space \cite{greenwald2001space}; recently, this algorithm was shown to be optimal by Cormode and Vesle\'{y} in 2020 \cite{cormode2020tight}. However, due to the intricacy of the GK sketch and its analysis, over-simplified versions of the algorithm are implemented in practical applications, often without any known theoretical guarantees. In fact, it has remained an open question whether the GK sketch can be simplified while maintaining the optimal space bound. In this paper, we resolve this open question by giving a simplified deterministic algorithm that stores at most $(2 + o(1))\epsilon^{-1} \log (\epsilon n)$ elements and solves the additive-error quantile estimation problem; as a side benefit, our algorithm achieves a smaller constant factor than the $\frac{11}{2} \epsilon^{-1} \log(\epsilon n)$ space bound in the original GK sketch~\cite{greenwald2001space}. Our algorithm features an easier analysis and still achieves the same optimal asymptotic space complexity as the original GK sketch.

Lastly, our simplification enables an efficient data structure implementation, with a worst-case runtime of $O(\log(1/\epsilon) + \log \log (\epsilon n))$ per-element for the ordinary $\epsilon$-approximate quantile estimation problem. Also, for the related ``weighted'' quantile estimation problem, we give efficient data structures for our simplified algorithm which guarantee a worst-case per-element runtime of $O(\log(1/\epsilon) + \log \log (\epsilon W_n/w_{\textrm{min}}))$, achieving an improvement over the previous upper bound of \cite{assadi2023generalizing}.