Differentially Private Hamiltonian Monte Carlo

Ossi Räisä, Antti Koskela, Antti Honkela

Keywords: Differential Privacy, Markov chain Monte Carlo, Hamiltonian Monte Carlo, Bayesian Inference

TL;DR: Differentially private Hamiltonian Monte Carlo using the Metropolis-Hastings acceptance test.

Abstract: We present DP-HMC, a variant of Hamiltonian Monte Carlo (HMC) that is differentially private (DP). We use the penalty algorithm of Yildirim and Ermis to make the acceptance test private, and add Gaussian noise to the gradients of the target distribution to make the HMC proposal private. Our main contribution is showing that DP-HMC has the correct invariant distribution, and is ergodic. We also compare DP-HMC with the existing penalty algorithm, as well as DP-SGLD and DP-SGNHT.

Paper Under Submission: The paper is currently under submission at NeurIPS