Selecting the Best in GANs Family: a Post Selection Inference Framework

Yao-Hung Hubert Tsai, Denny Wu, Makoto Yamada, Ruslan Salakhutdinov, Ichiro Takeuchi, Kenji Fukumizu

Abstract: "Which Generative Adversarial Networks (GANs) generates the most plausible images?" has been a frequently asked question among researchers. To address this problem, we first propose an \emph{incomplete} U-statistics estimate of maximum mean discrepancy $\textnormal{MMD}_{inc}$ to measure the distribution discrepancy between generated and real images. $\textnormal{MMD}_{inc}$ enjoys the advantages of asymptotic normality, computation efficiency, and model agnosticity. We then propose a GANs analysis framework to select the "best" member in GANs family using the Post Selection Inference (PSI) with $\textnormal{MMD}_{inc}$. In the experiments, we adopt the proposed framework on 7 GANs variants and compare their $\textnormal{MMD}_{inc}$ scores.

Keywords: Post Selection Inference, GAN Evaluation