Forward Explanation : Why Catastrophic Forgetting Occurs

Weimin Yin, Chunzhao Xie, Bin Chen, Zhenhao Tan

Primary Area: transfer learning, meta learning, and lifelong learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: catastrophic forgetting, interpretability, transfer learning, lifelong learning

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: We have fundamentally explained why neural networks experience catastrophic forgetting.

Abstract: The training framework relying on backpropagation and gradient descent has resulted in the creation of opaque models, leading to many problems that we cannot explain. One such problem that has remained inexplicable since the advent of neural networks is catastrophic forgetting. Recently, We have made some intriguing discoveries, which we have integrated into an explanation for neural network training, referred to as Forward Explanation. We first discover that training guides neural networks to produce a particular representation, which we refer to as Interleaved Representation. Additionally, we find that under this representation, neural networks exhibit a series of convergence phenomena, which we term Task Representation Convergence Phenomena. Furthermore, we find that in order to learn this representation, neural networks undergo a specific parameter change during training, which we call Forward-Interleaved Memory Encoding. This unveils some inner workings of how neural networks learn and fundamentally answers why catastrophic forgetting occurs.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 1856