What Do Visual Neural Networks Learn?

Daniella Har-Shalom, Yair Weiss

Abstract: Over the past decade, artificial neural networks trained to classify images downloaded from the internet have achieved astounding, almost superhuman performance and have been suggested as possible models for human vision. In this article, we review experimental evidence from multiple studies elucidating the classification strategy learned by successful visual neural networks (VNNs) and how this strategy may be related to human vision as well as previous approaches to computer vision. The studies we review evaluate the performance of VNNs on carefully designed tasks that are meant to tease out the cues they use. The use of this method shows that VNNs are often fooled by image changes to which human object recognition is largely invariant (e.g., the change of a few pixels in the image or a change of the background or illumination), and, conversely, that the networks can be invariant to very large image manipulations that disrupt human performance (e.g., randomly permuting the patches of an image). Taken together, the evidence suggests that these networks have learned relatively low-level cues that are extremely effective at classifying internet images but are ineffective at classifying many other images that humans can classify effortlessly.