Annealed Importance Sampling with q-Paths

Rob Brekelmans, Vaden Masrani, Thang D Bui, Frank Wood, Aram Galstyan, Greg Ver Steeg, Frank Nielsen

Keywords: annealed importance sampling, q-exponential family, non-extensive thermodynamics, information geometry

TL;DR: We generalize the geometric mixture path using power means related to the q-exponential family and alpha-divergence.

Abstract: Annealed importance sampling (AIS) is the gold standard for estimating partition functions or marginal likelihoods, corresponding to importance sampling over a path of distributions between a tractable base and an unnormalized target. While AIS yields an unbiased estimator for any path, existing literature has been limited to the geometric mixture or moment-averaged paths associated with the KL divergence and exponential family. We explore using $q$-paths for AIS, which are related to the homogeneous power means, deformed exponential family, and $\alpha$-divergence, and include the geometric path as a special case.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2012.07823/code)