Quality Evaluation of GANs Using Cross Local Intrinsic Dimensionality

Sukarna Barua, Xingjun Ma, Sarah Monazam Erfani, Michael Houle, James Bailey

Abstract: Generative Adversarial Networks (GANs) are an elegant mechanism for data generation. However, a key challenge when using GANs is how to best measure their ability to generate realistic data. In this paper, we demonstrate that an intrinsic dimensional characterization of the data space learned by a GAN model leads to an effective evaluation metric for GAN quality. In particular, we propose a new evaluation measure, CrossLID, that assesses the local intrinsic dimensionality (LID) of input data with respect to neighborhoods within GAN-generated samples. In experiments on 3 benchmark image datasets, we compare our proposed measure to several state-of-the-art evaluation metrics. Our experiments show that CrossLID is strongly correlated with sample quality, is sensitive to mode collapse, is robust to small-scale noise and image transformations, and can be applied in a model-free manner. Furthermore, we show how CrossLID can be used within the GAN training process to improve generation quality.

Keywords: Generative Adversarial Networks, Evaluation Metric, Local Intrinsic Dimensionality

TL;DR: We propose a new metric for evaluating GAN models.