Predictive coding feedback results in perceived illusory contours in a recurrent neural network

Zhaoyang Pang, Bhavin Choksi, Callum Biggs O'May, Rufin VanRullen

Keywords: deep learning, predictive coding, illusory contours, neuroscience

Abstract: Feedforward Convolutional Neural Networks (CNNs) have made great strides in solving computer vision tasks. However, these networks only roughly mimic human visual perception [1, 2]. For example, a recent report [3] shows that such neural networks do not perceive illusory contours (e.g., Kanizsa squares [4]) in the way that humans do. Physiological evidence suggests that the perception of illusory contours could involve feedback connections in the visual cortex [5, 6], which are lacking in feedforward networks [7, 8]. Would recurrent feedback neural networks perceive illusory contours like humans? In this work we address this issue by equipping a deep feedforward convolutional network with brain-inspired recurrent dynamics implementing a "predictive coding" strategy: at each layer of the hierarchical model, generative feedback “predicts” (or reconstructs) the pattern of activity in the previous layer, and the reconstruction errors are used to iteratively update the network’s representations across timesteps. The neural network was first pretrained on the CIFAR100 natural image dataset in an unsupervised way with a reconstruction objective. Then, a classification decision layer was added and the model was finetuned on a form discrimination task: squares vs. randomly oriented inducer shapes (no illusory contour). Finally, the model was tested with the unfamiliar "illusory contour" configuration: inducer shapes oriented to form an illusory square. Compared with the feedforward baseline, the iterative "predictive coding" feedback resulted in more illusory contours being classified as physical squares across timesteps. The illusory contour was measurable in the luminance profile of the image reconstructions produced by the model. In other words, the model was not merely confused by the novel configuration, but behaved as if a shape had truly been presented, similar to the human version of the illusion.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/predictive-coding-feedback-results-in/code)