Training GANs with Stronger Augmentations via Contrastive DiscriminatorDownload PDF

28 Sep 2020 (modified: 14 Jan 2021)ICLR 2021 PosterReaders: Everyone
  • Keywords: generative adversarial networks, contrastive learning, data augmentation, visual representation learning, unsupervised learning
  • Abstract: Recent works in Generative Adversarial Networks (GANs) are actively revisiting various data augmentation techniques as an effective way to prevent discriminator overfitting. It is still unclear, however, that which augmentations could actually improve GANs, and in particular, how to apply a wider range of augmentations in training. In this paper, we propose a novel way to address these questions by incorporating a recent contrastive representation learning scheme into the discriminator, coined ContraD. This "fusion" enables discriminators to work with much stronger augmentations without catastrophic forgetting, which can significantly improve GAN training. Even better, we observe that the contrastive learning itself also benefits from GAN training, i.e., keeping discriminative features between real and fake samples, suggesting a strong coherence between the two worlds: a good contrastive representation is also good for GAN discriminators, and vice versa. Our experimental results show that GAN with ContraD consistently improves FID scores compared to other recent techniques using data augmentations, still maintaining highly discriminative features in the discriminator in terms of the linear evaluation. Finally, as a byproduct, we show that our GANs trained in an unsupervised manner (without labels) can induce many conditional generative models via a simple latent sampling, leveraging the learned features of ContraD.
  • One-sentence Summary: We propose a novel discriminator of GAN showing that contrastive representation learning, e.g., SimCLR, and GAN can benefit each other when they are jointly trained.
  • Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics
17 Replies