Improving the Training of the GANs with Limited Data via Dual Adaptive Noise Injection

Zhaoyu Zhang, Yang Hua, Guanxiong Sun, Hui Wang, Seán F. McLoone

Abstract: Recently, many studies have highlighted that training Generative Adversarial Networks (GANs) with limited data suffers from the overfitting of the discriminator ($D$). Existing studies mitigate the overfitting of $D$ by employing data augmentation, model regularization, or pre-trained models. Despite the success of existing methods in training GANs with limited data, noise injection is another plausible, complementary, yet not well-explored approach to alleviate the overfitting of $D$ issue. In this paper, we propose a simple yet effective method called Dual Adaptive Noise Injection (DANI), to further improve the training of GANs with limited data. Specifically, DANI consists of two adaptive strategies: adaptive injection probability and adaptive noise strength. For the adaptive injection probability, Gaussian noise is injected into both real and fake images for generator ($G$) and $D$ with a probability $p$, respectively, where the probability $p$ is controlled by the overfitting degree of $D$. For the adaptive noise strength, the Gaussian noise is produced by applying the adaptive forward diffusion process to both real and fake images, respectively. As a result, DANI can effectively increase the overlap between the distributions of real and fake data during training, thus alleviating the overfitting of $D$ issue. Extensive experiments on several commonly-used datasets with both StyleGAN2 and FastGAN backbones demonstrate that DANI can further improve the training of GANs with limited data and achieve state-of-the-art results compared with other methods.

Primary Subject Area: [Generation] Multimedia Foundation Models

Secondary Subject Area: [Generation] Generative Multimedia

Relevance To Conference: This paper focuses on the fundamental research on training Generative Adversarial Networks (GANs) with limited data. In recent years, GANs have achieved great success in generating contents, e.g., images, videos, text and audio, for social media. These generated contents can be applied in various multimedia applications. Therefore, our research can inform the multimedia community, For example, it may ease the generation high quality multimedia content using only limited data.

Submission Number: 4399