Neural Darwinism: A Theoretical Framework for Representation Evolution in Convolutional Neural Networks

Boyuan Wang, Richard Jiang

Keywords: Neural Darwinism, Darwinian Score, Neural Darwinian Culling, Convolutional Neural Networks

Abstract: We introduce a Darwinian framework that mathematically formalizes representation evolution in deep networks, viewing each neuron as an adaptive entity competing for survival during training. In this perspective, learning is governed by a unified Darwinian Score that reflects three essential dimensions of neuronal fitness—informational diversity, functional contribution, and temporal adaptability. This score induces a principled constrained optimization objective that balances model compactness with predictive fidelity, supported by new approximation guarantees showing that preserving high-fitness neurons retains the network’s functional capacity. We then operationalize this framework through Neural Darwinism Culling (NDC), which serves as a practical instantiation of the Darwinian Score. NDC dynamically removes neurons with persistently low fitness while allowing high-value neurons to specialize. NDC captures the intrinsic evolutionary dynamics of neural representations: neurons with collapsed activations, negligible causal impact on loss reduction, or stagnant parameter trajectories are pruned, whereas differentiated and adaptable neurons are retained. This yields pruning decisions that are interpretable, layer-aware, and aligned with the competitive pressures naturally emerging across network depth. Experiments across diverse methodological settings demonstrate that NDC, as a direct application of the Darwinian Score, achieves substantially higher sparsity with improved generalization compared to SOTA methods, particularly under extreme compression. Ablations further confirm that the Darwinian Score is the key driver of these gains. Overall, our work provides both a general evolutionary lens for understanding representation dynamics and a practical, theory-grounded path toward efficient and adaptive deep learning.

Primary Area: learning theory

Submission Number: 5114