Internalized Biases in Fréchet Inception DistanceDownload PDF

Published: 02 Dec 2021, Last Modified: 05 May 2023NeurIPS 2021 Workshop DistShift PosterReaders: Everyone
Keywords: benchmarking, generative models, fid, gan
Abstract: Fréchet inception distance (FID) established itself as standard performance measuring method for generative adversarial networks (GANs). In this paper, we empirically investigate the biases that are inherited by its underlying design decision of extracting image features using the Inception v3 image classification network. As a result, we investigate how reliable FID is in terms of ranking performances of GANs. In this context, we find that FID is not aligned with human perception and exchanging Inception v3 with different image classification networks simply steers the ranking towards different biases.
TL;DR: Fréchet inception distance (FID) is inherently biased, because it is robust towards deviations in brightness, saturation, and contrast, but sensitive towards corruptions acting on edge and texture information.
1 Reply