High Probability Streaming Lower Bounds for $F_2$ Estimation

ICLR 2026 Conference Submission20365 Authors

Keywords: sketching, streaming, dimensionality reduction

Abstract: A recent paper of Braverman and Zamir [BZ'24] gave a lower bound of $\Omega(\frac{1}{\epsilon^2}\log n)$ for estimating the $F_2$ moment of a stream to within $1 \pm \epsilon$ multiplicative error, resolving the complexity of $F_2$ estimation for constant failure probability $\delta$ in the insertion-only model. We show that their argument can be adapted to achieve tight dependence on the failure probability $\delta$. Our key step is to replace the "Exam Set Disjointness" problem used in [BZ24] with a robust version that we call "Exam Mostly Frequency" (EMostlyFreq). This is the exam version of the communication problem underlying the high-probability analysis introduced in [Kamath, Price, Woodruff '21]. We prove a tight lower bound of $\Omega(\frac{1}{\epsilon^2} \log(\frac{\epsilon\sqrt{n}}{\log(1/\delta)}) \log(1/\delta))$ for $F_2$ estimation.

Primary Area: other topics in machine learning (i.e., none of the above)

Submission Number: 20365