Local Unsupervised Learning for Image Analysis

Leopold Grinberg, John Hopfield, Dmitry Krotov

Keywords: hebbian learning, local learning, orientation selectivity

Abstract: We use a recently proposed biologically plausible local unsupervised training algorithm (Krotov & Hopfield, PNAS 2019) for learning convolutional filters from CIFAR-10 images. These filters combined with patch normalization and very steep non-linearities result in a good classification accuracy for shallow networks trained locally, as opposed to end-to-end. The filters learned by our algorithm contain both orientation selective units and unoriented color units, resembling the responses of pyramidal neurons located in the cytochrome oxidase “interblob” and “blob” regions in the primary visual cortex of primates. It is shown that convolutional networks with patch normalization significantly outperform standard convolutional networks on the task of recovering the original classes when shadows are superimposed on top of standard CIFAR-10 images. Patch normalization approximates the retinal adaptation to the mean light intensity, important for human vision. All these results taken together suggest a possibility that local unsupervised training might be a useful tool for learning general representations (without specifying the task) directly from unlabeled data.