Optimal Variance Control of the Score Function Gradient Estimator for Importance Weighted Bounds

Valentin Liévin, Andrea Dittadi, Anders Christensen, Ole Wither

Abstract:

This paper introduces novel results for the score function gradient estimator of the
importance weighted variational bound (IWAE). We prove that in the limit of large
K (number of importance samples) one can choose the control variate such that the
Signal-to-Noise ratio (SNR) of the estimator grows as √K. This is in contrast to the
standard pathwise gradient estimator where the SNR decreases as 1/√K. Based on
our theoretical findings we develop a novel control variate that extends on VIMCO.
Empirically, for the training of both continuous and discrete generative models,
the proposed method yields superior variance reduction, resulting in an SNR
for IWAE that increases with K without relying on the reparameterization trick.
The novel estimator is competitive with state-of-the-art reparameterization-free
gradient estimators such as Reweighted Wake-Sleep (RWS) and the thermodynamic
variational objective (TVO) when training generative models