Distinguishing representational geometries with controversial stimuli: Bayesian experimental design and its application to face dissimilarity judgments

Tal Golan, Wenxuan Guo, Heiko H. Schütt, Nikolaus Kriegeskorte

Keywords: controversial stimuli, representational similarity analysis, face perception, neural networks, similarity judgments

TL;DR: We introduce a Bayesian controversial stimulus synthesis procedure that ensures contrasting representational geometry predictions by different candidate models. It uncovers behavioral evidence for an inverse-rendering model of human face perception.

Abstract: Comparing representations of complex stimuli in neural network layers to human brain representations or behavioral judgments can guide model development. However, even qualitatively distinct neural network models often predict similar representational geometries of typical stimulus sets. We propose a Bayesian experimental design approach to synthesizing stimulus sets for adjudicating among representational models. We apply our method to discriminate among alternative neural network models of behavioral face similarity judgments. Our results indicate that a neural network trained to invert a 3D-face-model graphics renderer is more human-aligned than the same architecture trained on identification, classification, or autoencoding. Our proposed stimulus synthesis objective is generally applicable to designing experiments to be analyzed by representational similarity analysis for model comparison.