Statistical Theory of Differentially Private Marginal-based Data Synthesis Algorithms

Ximing Li, Chendi Wang, Guang Cheng

Keywords: Synthetic data, differential privacy, marginal-based method, Bayesian network, learning theory

TL;DR: We analyze the differentially private marginal-based data synthesis algorithms in a statistical framework and establish a theoretical guarantee for the accuracy and utility.

Abstract:

Marginal-based methods achieve promising performance in the synthetic data competition hosted by the National Institute of Standards and Technology (NIST).
To deal with high-dimensional data, the distribution of synthetic data is represented by a probabilistic graphical model (e.g., a Bayesian network), while the raw data distribution is approximated by a collection of low-dimensional marginals.
Differential privacy (DP) is guaranteed by introducing random noise to each low-dimensional marginal distribution.
Despite its promising performance in practice, the statistical properties of marginal-based methods are rarely studied in the literature.
In this paper, we study DP data synthesis algorithms based on Bayesian networks (BN) from a statistical perspective. We establish a rigorous accuracy guarantee for BN-based algorithms, where the errors are measured by the total variation (TV) distance or the $L^2$ distance.
Related to downstream machine learning tasks, an upper bound for the utility error of the DP synthetic data is also derived. To complete the picture, we establish a lower bound for TV accuracy that holds for every $\epsilon$-DP synthetic data generator.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Social Aspects of Machine Learning (eg, AI safety, fairness, privacy, interpretability, human-AI interaction, ethics)