Variational Inference with Unnormalized Priors

TMLR Paper4197 Authors

Abstract: Energy-based models (EBMs) can be used as powerful priors for Variational Autoencoders (VAEs), improving latent space structure and generative performance. Although previous work has explored EBMs as priors in VAEs, training challenges remain, particularly in efficiently estimating or bypassing the intractable partition function. In this paper, we introduce a generalized training scheme for VAEs with EBM priors and present a comparative analysis of several instantiations of this framework, highlighting their impact on sample quality, likelihood estimation, and optimization stability. By addressing these challenges, we aim to advance the practical applicability of EBM-VAEs and offer insights into their theoretical foundations.

Submission Length: Regular submission (no more than 12 pages of main content)

Previous TMLR Submission Url: https://openreview.net/forum?id=zroaNP1o4V

Changes Since Last Submission: Re-evaluated the likelihood estimation procedure with a better implicit density ratio estimator that gives more reasonable lower-bounds. Also added extra binarized MNIST samples + latent space visualization in the appendix.

Assigned Action Editor: ~Jakub_Mikolaj_Tomczak1

Submission Number: 4197