Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning

Huiyi Wang, Haodong Lu, Lina Yao, Dong Gong

Keywords: Continual Learning, Parameter-efficient Fine-tuning, Self-expansion

Abstract:

Continual learning (CL) aims to continuously accumulate knowledge from non-stationary data streams without catastrophic forgetting of learned knowledge, requiring a balance between stability and plasticity. Leveraging generalizable representation in pre-trained models (PTMs), PTM-based CL methods adapt effectively to downstream tasks by adding learnable adapters or prompts to frozen PTMs. However, many existing methods restrict adaptation to a fixed set of modules, limiting CL capabilities. Periodically adding task-specific modules leads to linear model growth and impaired knowledge reuse. We propose **S**elf-**E**xpansion of PTMs with **M**odularized **A**daptation (SEMA), a novel approach that enhances stability-plasticity balance by automatically determining when to reuse or add adapter
modules depending on if distribution shifts that cannot be handled is detected at different representation levels. Our modular adapter consists of a functional adapter and a representation descriptor, which acts as a distribution shift indicator, triggering self-expansion. An expandable weighting router is learned jointly for mixture of adapter outputs. SEMA enables better knowledge reuse and sub-linear expansion rate. Extensive experiments show SEMA achieves state-of-the-art performance, outperforming PTM-based CL methods without memory rehearsal.

Submission Number: 8