Privacy Induces Robustness: Information-Computation Gaps and Sparse Mean EstimationDownload PDF

Published: 31 Oct 2022, Last Modified: 03 Jan 2023NeurIPS 2022 AcceptReaders: Everyone
Keywords: differential privacy, robustness, high-dimensional statistics, sparse mean estimation, sum-of-squares method, information-computation gaps, computational complexity of statistics, learning theory
TL;DR: New information-computation gaps for high-dimensional statistics arising due to privacy, and a new private algorithm for sparse mean estimation.
Abstract: We establish a simple connection between robust and differentially-private algorithms: private mechanisms *which perform well with very high probability* are automatically robust in the sense that they retain accuracy even if a constant fraction of the samples they receive are adversarially corrupted. Since optimal mechanisms typically achieve these high success probabilities, our results imply that optimal private mechanisms for many basic statistics problems are robust. We investigate the consequences of this observation for both algorithms and computational complexity across different statistical problems. Assuming the Brennan-Bresler secret-leakage planted clique conjecture, we demonstrate a fundamental tradeoff between computational efficiency, privacy leakage, and success probability for sparse mean estimation. Private algorithms which match this tradeoff are not yet known -- we achieve that (up to polylogarithmic factors) in a polynomially-large range of parameters via the Sum-of-Squares method. To establish an information-computation gap for sparse mean estimation, we also design new (exponential-time) mechanisms using fewer samples than efficient algorithms must use. Finally, we give evidence for privacy-induced information-computation gaps for several other statistics and learning problems, including PAC learning parity functions and estimation of the mean of a multivariate Gaussian.
Supplementary Material: pdf
8 Replies