NBSP: A Neuron-Level Framework for Balancing Stability and Plasticity in Deep Reinforcement Learning

ICLR 2026 Conference Submission15129 Authors

Keywords: deep reinforcement learning, stability-plasticity dilemma, skill neuron

TL;DR: We propose Neuron-level Balance between Stability and Plasticity (NBSP), a novel DRL framework that operates at the level of individual neurons to balance atability and plasticity.

Abstract: In contrast to the human ability to continuously acquire knowledge, agents struggle with the stability-plasticity dilemma in deep reinforcement learning (DRL), which refers to the trade-off between retaining existing skills (stability) and learning new knowledge (plasticity). Current methods focus on balancing these two aspects at the network level, lacking sufficient differentiation and fine-grained control of individual neurons. To overcome this limitation, we propose Neuron-level Balance between Stability and Plasticity (NBSP) method, by taking inspiration from the observation that specific neurons are strongly relevant to task-relevant skills. Specifically, NBSP first (1) defines and identifies RL skill neurons that are crucial for knowledge retention through a goal-oriented method, and then (2) introduces a framework by employing adaptive gradient masking and experience replay techniques targeting these neurons to preserve the encoded existing skills while enabling adaptation to new tasks. Numerous experimental results on the Meta-World, Atari, and DMC benchmarks demonstrate that NBSP significantly outperforms existing approaches in balancing stability and plasticity.

Primary Area: reinforcement learning

Submission Number: 15129