Progressive Development of the Number Sense in a Deep Neural Network

Will Y. Zou, James L. McClelland

Abstract: What are the developmental bases of the number sense? This ability could arise through evolution or experience. Stoianov & Zorzi [1] showed that a neural network could learn number sense from visual examples containing varying numbers of elements. However, the layer-wise training regime is unrealistic from a developmental standpoint. A key observation is that number acuity progressively develops from infancy to adulthood (as reflected by a decreasing Weber fraction). This development involves accumulation of single examples, each of which updates the connection weights in a hierarchical system. We present an unsupervised deep network that learns all weights as it observes one `number example’ at a time. As on-line training progresses, neurons representing numerosity start to emerge in the deeper layers, and the Weber fraction progressively sharpens. These results establish that a generic learning algorithm in a deep network gives rise to a clear developmental trajectory of the number sense.