The advantage of using Student's t-priors in variational autoencoders

Najmeh Abiri, Mattias Ohlsson

Keywords: Variational Autoencoders, DLVMs, Posterior Collapse

Abstract: Is it optimal to use the standard Gaussian prior in variational autoencoders? With Gaussian distributions, which are not weakly informative priors, variational autoencoders struggle to reconstruct the actual data. We provide numerical evidence that encourages using Student's t-distributions as default priors in variational autoencoders, and we challenge the usual setup for the variational autoencoder structure by comparing Gaussian and Student's t-distribution priors with different forms of the covariance matrix.