Metalearning Continual Learning Algorithms
Abstract: General-purpose learning systems should improve themselves in open-ended fashion in ever-changing environments. Conventional learning algorithms for neural networks, however, suffer from catastrophic forgetting (CF), i.e., previously acquired skills are forgotten when a new task is learned. Instead of hand-crafting new algorithms for avoiding CF, we propose Automated Continual Learning (ACL) to train self-referential neural networks to metalearn their own in-context continual (meta)learning algorithms. ACL encodes continual learning (CL) desiderata---good performance on both old and new tasks---into its metalearning objectives. Our experiments demonstrate that ACL effectively resolves "in-context catastrophic forgetting," a problem that naive in-context learning algorithms suffer from; ACL learned algorithms outperform both hand-crafted learning algorithms and popular meta-continual learning methods on the Split-MNIST benchmark in the replay-free setting, and enables continual learning of diverse tasks consisting of multiple standard image classification datasets. We also discuss the current limitations of in-context CL by comparing ACL with state-of-the-art CL methods that leverage pre-trained models. Overall, we bring several novel perspectives into the long-standing problem of CL.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: Please find the main edits highlighted in **violet** in the PDF.
The summary of changes is as follows:
- Figure 1 was updated to replace "train" by "demonstrations" and "test" by "query" to avoid confusions with meta-training and meta-testing. **Reviewer qEUp**
- Figure 1 caption updated accordingly. **Reviewers qEUp & NC5t**
- Sec 2. one paragraph added to introduce the background section **Reviewers qEUp & NC5t**
- Sec 2.2. improved the clarity on the metalearning/in-context learning (details regarding the sequence construction were moved from
Appendix A.2 to here). **Reviewer qEUp**
- Sec 3. the dependency on theta is made explicit in Eq 6 **Reviewer qEUp**
- Sec 4.1 one sentence added at the beginning to contrast the conventional catastrophic forgetting and its counterpart for in-context learning **Reviewer NC5t**
- Sec 4.1 an explanation added regarding the slight degradation on FC100+MiniImageNet case **Reviewer 5yTC**
- Sec 4.2 "learning" replaced by "metalearning" (same in Figure 2 captions). **Reviewer 5yTC**
- Table 3 caption: one sentence added regarding the joint training performance **Reviewer NC5t**
Further polishing has been done for the camera-ready version.
Code: https://github.com/IDSIA/automated-cl
Supplementary Material: zip
Assigned Action Editor: ~Alessandro_Sordoni1
Submission Number: 3534
Loading