New Lower Bounds for Private Estimation and a Generalized Fingerprinting LemmaDownload PDF

Published: 31 Oct 2022, Last Modified: 13 Jan 2023NeurIPS 2022 AcceptReaders: Everyone
Keywords: Differential Privacy, Learning, Machine Learning, Data Privacy, Statistics, Gaussians, Covariance Estimation, Lower Bounds, Mean Estimation
Abstract: We prove new lower bounds for statistical estimation tasks under the constraint of $(\varepsilon,\delta)$-differential privacy. First, we provide tight lower bounds for private covariance estimation of Gaussian distributions. We show that estimating the covariance matrix in Frobenius norm requires $\Omega(d^2)$ samples, and in spectral norm requires $\Omega(d^{3/2})$ samples, both matching upper bounds up to logarithmic factors. We prove these bounds via our main technical contribution, a broad generalization of the fingerprinting method to exponential families. Additionally, using the private Assouad method of Acharya, Sun, and Zhang, we show a tight $\Omega(d/(\alpha^2 \varepsilon))$ lower bound for estimating the mean of a distribution with bounded covariance to $\alpha$-error in $\ell_2$-distance. Prior known lower bounds for all these problems were either polynomially weaker or held under the stricter condition of $(\varepsilon,0)$-differential privacy.
TL;DR: We give new lower bounds for statistical estimation tasks, such as covariance estimation of Gaussians, under approximate differential privacy by proving a generalized fingerprinting lemma.
Supplementary Material: pdf
14 Replies