PixelVAE: A Latent Variable Model for Natural Images

Ishaan Gulrajani, Kundan Kumar, Faruk Ahmed, Adrien Ali Taiga, Francesco Visin, David Vazquez, Aaron Courville

Nov 04, 2016 (modified: Feb 23, 2017) ICLR 2017 conference submission readers: everyone
  • Abstract: Natural image modeling is a landmark challenge of unsupervised learning. Variational Autoencoders (VAEs) learn a useful latent representation and model global structure well but have difficulty capturing small details. PixelCNN models details very well, but lacks a latent code and is difficult to scale for capturing large structures. We present PixelVAE, a VAE model with an autoregressive decoder based on PixelCNN. Our model requires very few expensive autoregressive layers compared to PixelCNN and learns latent codes that are more compressed than a standard VAE while still capturing most non-trivial structure. Finally, we extend our model to a hierarchy of latent variables at different scales. Our model achieves state-of-the-art performance on binarized MNIST, competitive performance on 64 × 64 ImageNet, and high-quality samples on the LSUN bedrooms dataset.
  • TL;DR: VAE with an autoregressive PixelCNN-based decoder with strong performance on binarized MNIST, ImageNet 64x64, and LSUN bedrooms.
  • Conflicts: umontreal.ca, iitk.ac.in, polimi.it, cvc.uab.es
  • Authorids: igul222@gmail.com, kundankumar2510@gmail.com, faruk.ahmed.91@gmail.com, adrien.alitaiga@gmail.com, francesco.visin@polimi.it, dvazquez@cvc.uab.es, aaron.courville@gmail.com
  • Keywords: Deep learning, Unsupervised Learning