Spurious Features in Continual Learning

Timothee LESORT

Keywords: Spurious Features, Continual Learning, Plasticity

TL;DR: This paper show that catastrophic forgetting is partially due to spurious features.

Abstract:

Continual Learning (CL) is the research field addressing learning without forgetting when the data distribution is not static.
This paper studies spurious features' influence on continual learning algorithms.
We show that continual learning algorithms solve tasks by selecting features that are not generalizable.
Our experiments highlight that continual learning algorithms face two related problems: (1) spurious features (SP) and (2) local spurious features (LSP). The first one is due to a covariate shift between training and testing data, while the second is due to the limited access to data at each training step.
We study (1) through a consistent set of continual learning experiments varying spurious correlation amount and data distribution support.
We show that (2) is a major cause of performance decrease in continual learning along with catastrophic forgetting.
This paper presents a different way of understanding performance decrease in continual learning by highlighting the influence of (local) spurious features in algorithms capabilities.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning