Forgetting is Everywhere

Ben Sanati, Thomas L Lee, Trevor McInroe, Aidan Scannell, Nikolay Malkin, David Abel, Amos Storkey

Keywords: Forgetting, Bayesian learning, Reinforcement learning, Supervised learning

TL;DR: We propose and evaluate a formal definition of forgetting that unifies phenomena in supervised and reinforcement learning settings.

Abstract:

A fundamental challenge in developing general learning algorithms is their tendency to forget past knowledge when adapting to new data.
Addressing this problem requires a principled understanding of forgetting; yet, despite decades of study, no unified definition has emerged that provides insights into the underlying dynamics of learning.
We propose an algorithm- and task-agnostic theory that characterises forgetting as a lack of self-consistency in a learner's predictive distribution over future experiences, manifesting as a loss of predictive information.
Our theory naturally yields a general measure of an algorithm's propensity to forget.
To validate the theory, we design a comprehensive set of experiments that span classification, regression, generative modelling, and reinforcement learning.
Across these domains, we empirically demonstrate how forgetting is present across all learning settings and plays a significant role in determining learning efficiency.
Together, these results establish a principled understanding of forgetting and provide a foundation for analysing and improving the information retention capabilities of general learning algorithms.

Primary Area: probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)

Submission Number: 9120