Multiple Invertible and Equivariant Transformation for Disentanglement in VAEs

Hee-Jun Jung, Kangil Kim

Keywords: Variational AutoEncoder (VAE), Unsupervised Disentanglement Learning, Invertible and Equivariant function, Exponential Family

TL;DR: We improve disentangled representation learning with Multiple Invertible and Equivariant transformation (MIE-transformation) in VAEs.

Abstract: Disentanglement learning is a core issue for understanding and re-using trained information in Variational AutoEncoder (VAE), and effective inductive bias has been reported as a key factor. However, the actual implementation of such bias is still vague. In this paper, we propose a novel method, called MIE-transformation, to inject inductive bias by 1) guaranteeing the invertibility of latent-to-latent vector transformation while preserving a certain portion of equivariance of input-to-latent vector transformation, called IE-transformation, 2) extending the form of prior and posterior in VAE frameworks to an unrestricted form through a learnable conversion to an approximated exponential family, called EF-conversion, and 3) integrating multiple units of IE-transformation and EF-conversion, and their training. In experiments on 3D Cars, 3D Shapes, and dSprites datasets, MIE-transformation improves the disentanglement performance of state-of-the-art VAEs.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning