Slice Sampling Reparameterization Gradients

David M. Zoltowski, Diana Cai, Ryan P Adams

Keywords: Monte Carlo gradient estimation, slice sampling, energy-based models, variational inference

TL;DR: We present reparameterization gradients for samples generated from a slice sampler. We apply the method to optimize probabilistic objectives with expectations under unnormalized probability distributions.

Abstract:

Slice sampling is a Markov chain Monte Carlo algorithm for simulating samples from probability distributions, with the convenient property that it is rejection-free. When the slice endpoints are known, the sampling path is a deterministic function of noise variables since there are no accept-reject steps like those in Metropolis-Hastings algorithms. Here we describe how to differentiate the slice sampling path to compute slice sampling reparameterization gradients. Since slice sampling does not require a normalizing constant, this allows for computing reparameterization gradients of samples from potentially complicated multivariate distributions.
We apply the method in synthetic examples and to fit a variational autoencoder with a conditional energy-based model approximate posterior.