Random Quadratic Forms with Dependence: Applications to Restricted Isometry and Beyond

Arindam Banerjee, Qilong Gu, Vidyashankar Sivakumar, Zhiwei Steven Wu

Abstract: Several important families of computational and statistical results in machine learning and randomized algorithms rely on uniform bounds on quadratic forms of random vectors or matrices. Such results include the the Restricted Isometry Property (RIP), the Johnson-Lindenstrauss (J-L) Lemma, randomized sketching algorithms, and approximate linear algebra. The existing results critically depend on statistical independence, e.g., independent entries for random vectors, independent rows for random matrices, etc., which prevent their usage in adaptive modeling settings. In this paper, we show that such independence is in fact not needed for such results which continue to hold under fairly general dependence structures. In particular, we present uniform bounds on random quadratic forms of sub-Gaussian martingale difference sequences (MDSs) which allow general dependencies on the history. The results are applicable to adaptive modeling settings and also allows for sequential design of random vectors and matrices. We also discuss MDS based dependent design forms of RIP, J-L, and sketching, to illustrate the generality of the results.

CMT Num: 6848