Gaussian Density Parametrization Flow: Particle and Stochastic Approaches

Théo Galy-Fajou, Valerio Perrone, Manfred Opper

Keywords: variational inference, bayesian inference, gaussian, reparametrization

TL;DR: We derive a flow on the Gaussian reparametrization which can be solved both with stochastic approach or with particles.

Abstract:

Bayesian inference is intractable for most practical problems and requires approximation schemes with several trade-offs.
Variational inference provides one of such approximations which, while powerful, has thus far seen limited use in high-dimensional applications due to its complexity and computational cost. This paper introduces a scalable, theoretically-grounded, and simple-to-implement algorithm for approximate inference with a variational Gaussian distribution. Specifically, we establish a practical particle-based algorithm to perform variational Gaussian inference that scales linearly in the problem dimensionality. We show that our approach performs on par with the state of the art on a set of challenging high-dimensional problems.