Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics

Vinay Venkatesh Ramasesh, Ethan Dyer, Maithra Raghu

Keywords: Catastrophic forgetting, continual learning, representation analysis, representation learning

Abstract: Catastrophic forgetting is a recurring challenge to developing versatile deep learning models. Despite its ubiquity, there is limited understanding of its connections to neural network (hidden) representations and task semantics. In this paper, we address this important knowledge gap. Through quantitative analysis of neural representations, we find that deeper layers are disproportionately responsible for forgetting, with sequential training resulting in an erasure of earlier task representational subspaces. Methods to mitigate forgetting stabilize these deeper layers, but show diversity on precise effects, with some increasing feature reuse while others store task representations orthogonally, preventing interference. These insights also enable the development of an analytic argument and empirical picture relating forgetting to task semantic similarity, where we find that maximal forgetting occurs for task sequences with intermediate similarity.

One-sentence Summary: We study the layerwise change in representations due to catastrophic forgetting, and use our understanding to study how task similarity influences forgetting.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics