CM-GAN: Enhancing Consistency Model Image Quality and Stabilizing GAN Training
Abstract: Generative adversarial networks (GANs) have gained significant attention for generating realistic images, but they are notoriously difficult to train. In contrast, diffusion models provide stable training and avoid mode collapse, though their generation process is computationally intensive. To address this, Song et al. (2023) introduced consistency models (CMs), which optimize a novel consistency constraint derived from diffusion processes. In this paper, we propose a training method, CM-GAN, combining the strengths of both diffusion models and GANs while overcoming their respective limitations. We demonstrate that the same consistency constraint can be applied to stabilize GAN training and mitigate mode collapse. Meanwhile, CM-GAN serves as a fine-tuning mechanism for CMs by leveraging a discriminator, resulting in superior performance compared to CMs alone. Empirical results on benchmarks such as ImageNet 64$\times$64 and Bedroom 256$\times$256 show that CM-GAN significantly enhances the sample quality of CMs and effectively stabilizes GAN training.
Submission Length: Regular submission (no more than 12 pages of main content)
Assigned Action Editor: ~Jaesik_Park3
Submission Number: 3820
Loading