What cannot be learnt by Variational Autoencoder?

Anonymous

Nov 07, 2017 (modified: Dec 11, 2017) ICLR 2018 Conference Blind Submission readers: everyone Show Bibtex
  • Abstract: What would be learned variational autoencoder(VAE) and what influence the disentanglement of VAE? This paper tries to preliminarily address those issues theoretically in the idealistic situation and practically through noise modeling perspective in the realistic case. Information conservation (e.g. 2 Gaussian and 3 Gaussian cannot be the generating factors of each other), factor equivalence (e.g. the orthogonal transformation of Gaussian factors is still Gaussian factors) and separation of mutual information properties in the idealistic VAE case are revealed. They preliminary but fundamentally answer what would be learned by idealistic VAE and what cannot. Several performance indicators regarding the disentanglement and generating influence are subsequently raised to evaluate the performance of VAE model and to supervise the used factors. On other fold, the implementations under noise modeling empirically testify the theoretical analysis and also show their own characteristic in pursuing disentanglement.
  • TL;DR: This paper tries to preliminarily address the disentanglement and reconstruction issues theoretically in the idealistic situation and practically through noise modelling perspective in the realistic case.
  • Keywords: variational autoencoder, information theory, noise modelling, representation learning, generative model, disentanglement

Loading