On the Statistical Complexity of Estimation and Testing under Privacy Constraints
Abstract: The challenge of producing accurate statistics while respecting the privacy of the individuals in a sample is an important area of research. We study minimax lower bounds for classes of differentially private estimators. In particular, we show how to characterize the power of a statistical test under differential privacy in a plug-and-play fashion by solving an appropriate transport problem. With specific coupling constructions, this observation allows us to derive Le Cam-type and Fano-type inequalities not only for regular definitions of differential privacy but also for those based on Renyi divergence. We then proceed to illustrate our results on three simple, fully worked out examples. In particular, we show that the problem class has a huge importance on the provable degradation of utility due to privacy. In certain scenarios, we show that maintaining privacy results in a noticeable reduction in performance only when the level of privacy protection is very high. Conversely, for other problems, even a modest level of privacy protection can lead to a significant decrease in performance. Finally, we demonstrate that the DP-SGLD algorithm, a private convex solver, can be employed for maximum likelihood estimation with a high degree of confidence, as it provides near-optimal results with respect to both the size of the sample and the level of privacy protection. This algorithm is applicable to a broad range of parametric estimation procedures, including exponential families.
Submission Length: Long submission (more than 12 pages of main content)
Changes Since Last Submission: The changes compared to the previous version are the following :
(1) We swapped the template from the anonymous one to the camera ready one.
(2) We fixed the typos that were pointed out in the review and a few others.
(2) We emphasized the "plug and play" approach in the abstract and in the introduction.
(3) We reduced the discussion on the constants.
(4) We changed the notation $M \text{ s.t. } \mathcal{C}$ for the set one $M \in \mathcal{C}$, with a sentence explaining it.
(5) We moved the parts that were suggested to be moved to the appendix to the appendix, and we added small paragraphs that point to them for context.
(6) We added a small paragraph about non i.i.d. distributions in Section 3.
(7) We added the acknowledgements at the end of the article, which in particular include a sentence to thank the anonymous reviewers of this article.
Assigned Action Editor: ~Nihar_B_Shah1
License: Creative Commons Attribution 4.0 International (CC BY 4.0)
Submission Number: 753
Loading