Incremental Learning with Task-Specific Adapters
Keywords: Adaptors, Incremental Learning, Computer Vision, Transfer Learning
Abstract: Incremental learning aims to continuously acquire new knowledge while preserving previously learned information. Existing literature primarily focuses on improving model stability, often at the cost of plasticity, to prevent the forgetting of earlier tasks. In this paper, we argue that inter-task differences are the primary driver of catastrophic forgetting. To address this challenge, we propose a novel network architecture compromising two distinct components: one dedicated to learning invariant features shared across tasks and another for capturing task-specific details. Specifically, we repurpose adapters, originally introduced for parameter-efficient fine-tuning, as feature modifiers to capture task-specific details, while the backbone network focuses on learning invariant features. Unlike prior approaches that keep the backbone frozen and only fine-tune {adapters}, we co-train both the backbone network and adapters, employing an additional regularization term that encourages the backbone to learn shared features. Our approach integrates seamlessly with established methods, such as Learning without Forgetting (LwF). Extensive experiments on CIFAR-100 and ImageNet datasets demonstrate that our adapter-based methods consistently outperform non-adapter counterparts across diverse learning scenarios, including various task orders and data scales. Our approach improves both plasticity and stability, effectively addressing the stability-plasticity dilemma.
Primary Area: transfer learning, meta learning, and lifelong learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7134
Loading