Local Ridge Regression Resets Mitigate Plasticity Loss

Nitin Jain, Georg Martius, Marin Vlastelica

Track: Full track

Keywords: Plasticity Loss, Continual Learning, Reinforcement Learning

TL;DR: We propose Ridge Regression Reset, a novel approach to mitigate plasticity loss in Reinforcement Learning which addresses problems with existing approaches.

Abstract: Plasticity loss refers to a neural network's diminishing ability to learn in non-stationary environments. In Reinforcement Learning (RL), existing plasticity loss mitigation methods like full network resets, Plasticity Injection, and ReDo offer partial solutions to this problem, but are limited by issues such as catastrophic performance collapse and computational inefficiency. This paper introduces Ridge Regression Reset (R3), a novel approach that maintains output stability while restoring plasticity through an optimization framework. Our experiments show that R3 effectively mitigates plasticity loss, avoids catastrophic performance collapses, and provides better sample efficiency.

Submission Number: 17