On Sketching Quadratic Forms

Alexandr Andoni, Jiecao Chen, Robert Krauthgamer, Bo Qin, David P. Woodruff, Qin Zhang

Abstract: We undertake a systematic study of sketching a quadratic form: given an n x n matrix A, create a succinct sketch sk(A) which can produce (without further access to A) a multiplicative (1+ε)-approximation to xT A x for any desired query x ∈ Rn. While a general matrix does not admit non-trivial sketches, positive semi-definite (PSD) matrices admit sketches of size θ(ε{-2 n), via the Johnson-Lindenstrauss lemma, achieving the "for each" guarantee, namely, for each query x, with a constant probability the sketch succeeds. (For the stronger "for all" guarantee, where the sketch succeeds for all x's simultaneously, again there are no non-trivial sketches.) We design significantly better sketches for the important subclass of graph Laplacian matrices, which we also extend to symmetric diagonally dominant matrices. A sequence of work culminating in that of Batson, Spielman, and Srivastava (SIAM Review, 2014), shows that by choosing and reweighting O(ε{-2 n) edges in a graph, one achieves the "for all" guarantee. Our main results advance this front. For the "for all" guarantee, we prove that Batson et al.'s bound is optimal even when we restrict to "cut queries" x ∈ (0,1}n. Specifically, an arbitrary sketch that can (1+ε)-estimate the weight of all cuts (S, bar S) in an n-vertex graph must be of size Ω(ε{-2 n) bits. Furthermore, if the sketch is a cut-sparsifier (i.e., itself a weighted graph and the estimate is the weight of the corresponding cut in this graph), then the sketch must have Ω(ε{-2 n) edges. In contrast, previous lower bounds showed the bound only for spectral-sparsifiers. For the "for each" guarantee, we design a sketch of size Õ(ε{-1 n) bits for "cut queries" x ∈{0,1}n. We apply this sketch to design an algorithm for the distributed minimum cut problem. We prove a nearly-matching lower bound of Ω(ε{-1 n) bits. For general queries x ∈ Rn, we construct sketches of size Õ(ε{-1.6 n) bits. Our results provide the first separation between the sketch size needed for the "for all" and "for each" guarantees for Laplacian matrices.