Bayesian inference via sparse Hamiltonian flows

Naitong Chen, Zuheng Xu, Trevor Campbell

Keywords: Bayes, coresets, flows, Hamiltonian

TL;DR: This paper shows how to (1) construct Bayesian coresets simply and tractably using variational flows, and (2) make variational flows cheaper in the large-data regime via coresets.

Abstract: A Bayesian coreset is a small, weighted subset of data that replaces the full dataset during Bayesian inference, with the goal of reducing computational cost. Although past work has shown empirically that there often exists a coreset with low inferential error, efficiently constructing such a coreset remains a challenge. Current methods tend to be slow, require a secondary inference step after coreset construction, and do not provide bounds on the data marginal evidence. In this work, we introduce a new method---sparse Hamiltonian flows---that addresses all three of these challenges. The method involves first subsampling the data uniformly, and then optimizing a Hamiltonian flow parametrized by coreset weights and including periodic momentum quasi-refreshment steps. Theoretical results show that the method enables an exponential compression of the dataset in a representative model, and that the quasi-refreshment steps reduce the KL divergence to the target. Real and synthetic experiments demonstrate that sparse Hamiltonian flows provide accurate posterior approximations with significantly reduced runtime compared with competing dynamical-system-based inference methods.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2203.05723/code)