Information-theoretic stochastic contrastive conditional GAN: InfoSCC-GAN

Vitaliy Kinakh, Mariia Drozdova, Guillaume Quétant, Svyatoslav Voloshynovskyy, Tobias GOLLING

Keywords: GANs, Generative adversarial networks, Contrastive learning, Conditional image generation

Abstract:

Conditional generation is a subclass of generative problems when the output of generation is conditioned by a class attributes’ information. In this paper, we present a new stochastic contrastive conditional generative adversarial network (InfoSCC-GAN) with explorable latent space. The InfoSCC-GAN architecture is based on an unsupervised contrastive encoder built on the InfoNCE paradigm, attributes' classifier, and stochastic EigenGAN generator.
We propose two approaches for selecting the class attributes: external attributes from the dataset annotations and internal attributes from the clustered latent space of the encoder. We propose a novel training method based on a generator regularization using external or internal attributes every $n$-th iteration using the pre-trained contrastive encoder and pre-trained attributes’ classifier. The proposed InfoSCC-GAN is derived from an information-theoretic formulation of mutual information maximization between the input data and latent space representation for the encoder and the latent space and generated data for the decoder. Thus, we demonstrate a link between the training objective functions and the above information-theoretic formulation. The experimental results show that InfoSCC-GAN outperforms vanilla EigenGAN in image generation on several popular datasets, yet providing an interpretable latent space. In addition, we investigate the impact of regularization techniques and each part of the system by performing an ablation study. Finally, we demonstrate that thanks to the stochastic EigenGAN generator, the proposed framework enjoys a truly stochastic generation in contrast to vanilla deterministic GANs yet with the independent training of an encoder, a classifier, and a generator.
The code, supplementary materials, and demos are available \url{https://anonymous.4open.science/r/InfoSCC-GAN-D113}

One-sentence Summary: We propose a novel approach for conditional image generation, based on independent pretrained encoder, independent pretrained classifier and generator with explorable latent space

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2112.09653/code)