Manifold Learning and Alignment with Generative Adversarial Networks

Jiseob Kim, Seungjae Jung, Hyundo Lee, Byoung-Tak Zhang

Keywords: Generative Adversarial Networks, Manifold Learning, Manifold Alignment

TL;DR: We present a generative adversarial network that performs both multi-manifold learning and manifold alignment, utilizing the abstraction ability of encoder architecture.

Abstract: We present a generative adversarial network (GAN) that conducts manifold learning and alignment (MLA): A task to learn the multi-manifold structure underlying data and to align those manifolds without any correspondence information. Our main idea is to exploit the powerful abstraction ability of encoder architecture. Specifically, we define multiple generators to model multiple manifolds, but in a particular way that their inverse maps can be commonly represented by a single smooth encoder. Then, the abstraction ability of the encoder enforces semantic similarities between the generators and gives a plausibly aligned embedding in the latent space. In experiments with MNIST, 3D-Chair, and UT-Zap50k datasets, we demonstrate the superiority of our model in learning the manifolds by FID scores and in aligning the manifolds by disentanglement scores. Furthermore, by virtue of the abstractive modeling, we show that our model can generate data from an untrained manifold, which is unique to our model.

Code: https://www.dropbox.com/s/6odr268ybngcapm/iclr2020_mla_gan.zip?dl=0