Deep automodulators

Ari Heljakka, Yuxin Hou, Juho Kannala, Arno Solin

TL;DR: A novel autoencoder model that supports mutually independent decoder layers, enabling e.g. style mixing.

Abstract: We introduce a novel autoencoder model that deviates from traditional autoencoders by using the full latent vector to independently modulate each layer in the decoder. We demonstrate how such an 'automodulator' allows for a principled approach to enforce latent space disentanglement, mixing of latent codes, and a straightforward way to utilize prior information that can be construed as a scale-specific invariance. Unlike GANs, autoencoder models can directly operate on new real input samples. This makes our model directly suitable for applications involving real-world inputs. As the architectural backbone, we extend recent generative autoencoder models that retain input identity and image sharpness at high resolutions better than VAEs. We show that our model achieves state-of-the-art latent space disentanglement and achieves high quality and diversity of output samples, as well as faithfulness of reconstructions.

Keywords: unsupervised learning, generative models, autoencoders, disentanglement, style transfer

Community Implementations: [ 7 code implementations](https://www.catalyzex.com/paper/arxiv:1912.10321/code)