High-Dimensional Geometric Streaming for Nearly Low Rank Data

Hossein Esfandiari, Praneeth Kacham, Vahab Mirrokni, David Woodruff, Peilin Zhong

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Coresets, Subspace Approximation, Streaming Algorithms

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

We study streaming algorithms for the outer $(d-k)$-radius estimation of a set of points $a_1, \ldots ,a_n \in \mathbb{R}^d$. The problem asks to compute the minimum over all $k$-dimensional flats $F$ of $\max_i d(a_i, F)$, where $d(u, F)$ denotes the distance of a point $u$ from the flat $F$. This problem has been extensively studied in earlier works (Varadarajan et al., SIAM J. Comput. 2006) over a wide range of values of $d$, $k$ and $d-k$. The earlier algorithms are based on SDP relaxations of the problem and are not applicable in the streaming setting where we do not have space to store all the rows that we see. We give an efficient streaming coreset algorithm that selects $\text{poly}(k, \log n)$ rows and at the end outputs a $\text{poly}(k, \log n)$ approximation to the outer $(d-k)$-radius. The algorithm only uses $d \cdot \text{poly}(k, \log n)$ bits of space and runs in an overall time of $O(\text{nnz}(A) \cdot \log n + \text{poly}(d, \log n))$, where $\text{nnz}(A)$ denotes the number of nonzero entries in the $n \times d$ matrix $A$ with rows given by $a_1, \ldots, a_n \in \mathbb{R}^d$.

In a recent work, Woodruff and Yasuda (FOCS 2022), give streaming algorithms for a number of high-dimensional geometric problems such as width estimation, convex hull estimation, volume estimation etc. Their algorithms require $\Omega(d^2)$ bits of space and have an $\Omega(\sqrt{d})$ multiplicative approximation factor even when the rows $a_1,\ldots, a_n$ are “almost” spanned by a $k$ dimensional subspace. We show that when the rows are $a_1,\ldots,a_n$ are “almost” spanned by a $k$ dimensional space, our streaming coreset construction algorithm can be used to obtain algorithms that use only $O(d \cdot \text{poly}(k, \log n))$ bits of space and have a multiplicative error of $O(\text{poly}(k, \log n))$. When $d$ is large and $k$ is much smaller than $d$, our algorithms use a much smaller amount of space while guaranteeing a better approximation. We pay an additive error depending on how close the rows $a_1,\ldots,a_n$ to being spanned by a rank $k$ subspace.

As another application of our algorithm, we show that our streaming coreset can also be used to obtain approximations to the $\ell_p$ subspace approximation problem using exponential random variables to embed the $\ell_p$ subspace approximation problem into an instance of the $\ell_{\infty}$ subspace approximation problem.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 5979