Face Synthesis via User Manipulation of Disentangled Latent Representation

Nao Nakagawa, Ren Togo, Takahiro Ogawa, Miki Haseyama

Abstract: In this paper, we propose a novel face synthesis method whose process can be conditioned not only on labels but also on latent variables as “unknown” labels corresponding to unrevealed factors. We extend Variational Autoencoder (VAE) without any additional networks to introduce conditional generation, disentangled representation, and adversarial learning into one autoencoder. Since previous conditional generative models require the annotation of labels to condition them on, disentanglement, i.e., the unsupervised discovery of generative factors enables users to generate face images more flexibly and more efficiently. Moreover, although generative adversarial networks (GANs) have problems of mode collapse and instability of the learning process, adversarial learning on VAE in an introspective way achieves both the variation of results and the stability of generation. Evaluations on the CelebFaces Attributes Dataset (CelebA) show that our method can generate face images following users' conditioning both on the known and the “unknown” labels.