Resource-efficient perceptron has sparse synaptic weight distribution

Cengiz Pehlevan, Anirvan M. Sengupta

Abstract: Resource-efficiency is important for biological function of neurons. Using the perceptron as a model of a neuron, we show that resource-efficient learning implies sparse neural connectivity. The perceptron associates inputs to outputs by adjusting its synaptic weights. The learned synaptic weights are proposed to be the most resource-efficient by minimizing a biological resource cost given by the total absolute synaptic weight (l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -norm). Analytical methods from statistical physics and numerical simulations demonstrate that a resource-efficient perceptron has sparse connectivity. Sparseness decreases and resource usage increases with the number of associations to be learned. Our results have implications for synaptic connectivity in the cerebellum, where supervised learning is believed to happen.