Exploring a novel Feedback Mechanism for Convolutional Neural Networks

TMLR Paper5446 Authors

Abstract: Convolutional neural networks (CNNs), which have achieved significant success in various visual tasks, are inspired by the architecture of the mammalian vision system. However, unlike CNNs, the visual cortex contains a substantial number of top-down or feedback connections. Inspired by this, recent research has investigated incorporating feedback mechanisms into CNNs. In this paper, we propose a novel feedback mechanism called 'Image Specific Feature Selection (ISFS)' that leverages feedback to utilize only a relevant subset of filters for the given image. The feedback weights are learned, and thus the network learns to select features/filters tailored to each image. The feedback improves performance both in terms of better accuracy and better confidence in classification. The selection of filters through the feedback is indeed image-specific and results in interesting behaviour of the network. The feedback signals produced for a given image, can be viewed as a useful low-dimensional approximation of the internal representation of the image. We demonstrate that we can effectively use the feedback signals to identify when a given image has adversarial noise.

Submission Length: Long submission (more than 12 pages of main content)

Assigned Action Editor: ~Gang_Niu1

Submission Number: 5446