Variationally Inferred Sampling Through a Refined Bound

Victor Gallego, David Rios Insua

Keywords: SG-MCMC, variational approximation, particle flow

Abstract: A framework for efficient Bayesian inference in probabilistic programs is introduced by embedding a sampler inside a variational posterior approximation. Its strength lies in both ease of implementation and automatically tuning sampler parameters to speed up mixing time. Several strategies to approximate the evidence lower bound (ELBO) computation are introduced, including a rewriting of the ELBO objective. Experimental evidence is shown by performing experiments on an unconditional VAE on density estimation tasks; solving an influence diagram in a high-dimensional space with a conditional variational autoencoder (cVAE) as a deep Bayes classifier; and state-space models for time-series data.

TL;DR: We embed SG-MCMC samplers inside a variational approximation