Epanechnikov Variational Autoencoder

TMLR Paper4192 Authors

Abstract: In this paper, we bridge Variational Autoencoders (VAEs) and kernel density estimations (KDEs) by approximating the posterior by KDEs and deriving a new lower bound of empirical log likelihood. The f lexibility of KDEs not only addresses the limitations of Gaussian latent space in vanilla VAE but also provides a new perspective of estimating the KL-divergence term in original evidence lower bound (ELBO). We then propose the Epanechnikov kernel based VAE and show that the Epanechnikov kernel gives the tightest upper bound in estimating the KLdivergence under appropriate conditions in theory. Compared with Gaussian kernel, Epanechnikov kernel has compact support which should make the generated sample less blurry. The implementation of Epanechnikov kernel in VAE is straightforward as it lies in the "location-scale" family of distributions where reparametrization tricks can be applied directly. A series of experiments on benchmark datasets such as MNIST, Fashion-MNIST, CIFAR-10 and CelebA illustrate the superiority of Epanechnikov Variational Autoenocoder (EVAE) over vanilla VAE and other baseline models in the quality of reconstructed images, as measured by the FID score and Sharpness.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

* Corrected typos, mistakes and inconsistency of notations/reference format proposed by reviewers.
* Rewrote introduction part with more current literatures to situate the paper in modern machine learning and make the problems our method addresses clear.

Assigned Action Editor: ~Gabriel_Loaiza-Ganem1

Submission Number: 4192