Continual Release Moment Estimation with Differential Privacy

Nikita Kalinin, Jalaj Upadhyay, Christoph H. Lampert

Keywords: Differential Privacy, Continual Release, Moment Estimation

TL;DR: A method for continually and privately estimating both the first and second moments of a data stream with reduced noise compared to naive approaches.

Abstract: We propose *Joint Moment Estimation* (JME), a method for continually and privately estimating both the first and second moments of a data stream with reduced noise compared to naive approaches. JME supports the *matrix mechanism* and exploits a joint sensitivity analysis to identify a privacy regime in which the second-moment estimation incurs no additional privacy cost, thereby improving accuracy while maintaining privacy. We demonstrate JME’s effectiveness in two applications: estimating the running mean and covariance matrix for Gaussian density estimation and model training with DP-Adam.

Primary Area: Social and economic aspects of machine learning (e.g., fairness, interpretability, human-AI interaction, privacy, safety, strategic behavior)

Submission Number: 11815