Why are Modern GANs Poor Density Models?

Roy Friedman, Yair Weiss

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: GANs, density estimation, prior

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract:

Modern generative adversarial networks (GANs) generate extremely realistic images and are generally believed to capture the true data distribution. In this work, we evaluate modern GANs as density models and ask whether they can be used for tasks such as outlier detection and generative classification. We find that the performance of state-of-the-art GANs is very poor on these tasks and is often close to (or worse than) random. For instance, a modern GAN that generates remarkably realistic samples when trained on CIFAR10, consistently assigns higher likelihood to flat images than to images from the training set.
To try and understand the source of this poor performance, we show that the likelihood that a GAN assigns to an input image is dominated by the quality of the GAN reconstruction. Surprisingly, GANs often fail to reconstruct images from the training set while they are highly effective at reconstructing images outside the distribution. Taken together, our results indicate that modern GANs do not truly learn the underlying distribution, despite the impressive quality of the generated samples.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 3389