On Consistent Bayesian Inference from Synthetic Data

Ossi Räisä, Joonas Jälkö, Antti Honkela

Keywords: synthetic data, Bayesian inference, Bernstein-von Mises theorem, differential privacy

TL;DR: We prove that consistent Bayesian inference from synthetic data reusing existing samplers is possible with multiple large synthetic datasets.

Abstract: Generating synthetic data, with or without differential privacy, has attracted significant attention as a potential solution to the dilemma between making data easily available, and the privacy of data subjects. Several works have shown that consistency of downstream analyses from synthetic data, including accurate uncertainty estimation, requires accounting for the synthetic data generation. There are very few methods of doing so, most of them for frequentist analysis. In this paper, we study how to perform consistent Bayesian inference from synthetic data. We prove that mixing posterior samples obtained separately from multiple large synthetic datasets converges to the posterior of the downstream analysis under standard regularity conditions when the analyst's model is compatible with the data provider's model. We show experimentally that this works in practice, unlocking consistent Bayesian inference from synthetic data while reusing existing downstream analysis methods.

Submission Number: 6883