Comparing the local information geometry of image representations

David Lipshutz, Jenelle Feather, Sarah E Harvey, Alex H Williams, Eero P Simoncelli

Track: Extended Abstract Track

Keywords: representational similarity metric; Fisher information; information geometry; perception

TL;DR: We propose a metric for comparing the local geometry of image representations and use it to synthesize image distortions that separate models.

Abstract: We propose a framework for comparing a set of image representations (artificial or biological) in terms of their sensitivities to local distortions. We quantify the local geometry of a representation using the Fisher information matrix (FIM), a standard statistical tool for characterizing the sensitivity to local distortions of a stimulus, and use this as a substrate for a metric on the local geometry of representations in the vicinity of a base image. This metric may then be used to optimally differentiate a set of models, by optimizing for a pair of distortions that maximize the variance of the models under this metric. We use the framework to compare a set of simple models of the early visual system, identifying a novel set of image distortions that allow immediate comparison of the models by visual inspection. In a second example, we show that the method can reveal distinctions between standard and adversarially trained object recognition networks.

Submission Number: 51