Learning Continually at Peak Performance with Continuous Continual Backpropagation

ICLR 2026 Conference Submission24844 Authors

Keywords: Continual Learning, Reinforcement Learning, Plasticity, Dormant Neurons, Optimizer, Continual Reinforcement Learning

TL;DR: We introduce Continuous Continual Backpropagation (CCBP), which introduces utility-scaled partial resets inplace of full neurons to maintain peak performance throughout Continual Reinforcement Learning

Abstract: Training neural networks under non-stationary data distributions, as in continual supervised and reinforcement learning, is hindered by loss of plasticity and representation collapse. While recent approaches employ periodic, full neuron reinitialization to sustain gradient flow and restore plasticity, they often sacrifice asymptotic performance and still suffer frequent collapse. To address these limitations, we propose Continuous Continual Backpropagation (CCBP), which instead continuously, partially resets units. Empirically, CCBP preserves the long-term performance of standard optimizers while maintaining the plasticity of reset-based methods, and uniquely prevents policy collapse. Ablations further show how CCBP can be tuned to smoothly trade off plasticity and asymptotic performance, highlighting gradual reinitialization as a promising direction for continual deep learning.

Primary Area: reinforcement learning

Submission Number: 24844