Robust Max Entrywise Error Bounds for Tensor Estimation From Sparse Observations via Similarity-Based Collaborative Filtering

Devavrat Shah, Christina Lee Yu

Abstract: Consider the task of estimating a 3-order http://www.w3.org/1998/Math/MathML" xmlns:xlink="" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink"> $n \times n \times n$ tensor from noisy observations of randomly chosen entries in the sparse regime. We introduce a similarity based collaborative filtering algorithm for estimating a tensor from sparse observations and argue that it achieves sample complexity that nearly matches the conjectured computationally efficient lower bound on the sample complexity for the setting of low-rank tensors. Our algorithm uses the matrix obtained from the flattened tensor to compute similarity, and estimates the tensor entries using a nearest neighbor estimator. We prove that the algorithm recovers a finite rank tensor with maximum entry-wise error (MEE) and mean-squared-error (MSE) decaying to 0 as long as each entry is observed independently with probability http://www.w3.org/1998/Math/MathML" xmlns:xlink="" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink"> $p = \Omega (n^{-3/2 + \kappa })$ for any arbitrarily small http://www.w3.org/1998/Math/MathML" xmlns:xlink="" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink"> $ \kappa > 0$ . More generally, we establish robustness of the estimator, showing that when arbitrary noise bounded by http://www.w3.org/1998/Math/MathML" xmlns:xlink="" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink"> $ \boldsymbol { \varepsilon }\geq 0$ is added to each observation, the estimation error with respect to MEE and MSE degrades by http://www.w3.org/1998/Math/MathML" xmlns:xlink="" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink"> ${\sf poly}(\boldsymbol { \varepsilon })$ . Consequently, even if the tensor may not have finite rank but can be approximated within http://www.w3.org/1998/Math/MathML" xmlns:xlink="" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink"> $ \boldsymbol { \varepsilon }\geq 0$ by a finite rank tensor, then the estimation error converges to http://www.w3.org/1998/Math/MathML" xmlns:xlink="" target="_blank" rel="nofollow">http://www.w3.org/1999/xlink"> ${\sf poly}(\boldsymbol { \varepsilon })$ . Our analysis sheds insight into the conjectured sample complexity lower bound, showing that it matches the connectivity threshold of the graph used by our algorithm for estimating similarity between coordinates.