Learning to Prompt for Continual Learning

Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas Pfister

Keywords: Continual learning, Prompt-based learning

Abstract: The mainstream learning paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. This work explores a new paradigm for continual learning -- learning to dynamically prompt the model to learn tasks sequentially under different task transitions. Specifically, our method, Learning to Prompt for Continual Learning (L2P), prepends a subset of learnable parameters (called Prompts) from a larger set (called Prompt Pool) to the input embeddings. The training objective is designed to dynamically select and update prompts from the prompt pool to learn tasks sequentially given a pretrained backbone model. Under our new framework, instead of mitigating catastrophic forgetting via adapting large model parameters as in the previous continual learning paradigm, we tackle the problem of learning better small prompt parameters. In this framework, the prompt pool explicitly manages task-invariant and task-specific knowledge while maintaining model plasticity. The proposed L2P outperforms previous work in terms of forgetting on all datasets, including rehearsal-based methods on certain benchmarks, with privacy benefits from not requiring access to the data of previous tasks. Moreover, when L2P is additionally equipped with a rehearsal buffer, it matches the performance of training all tasks together, which is often regarded as an upper bound in continual learning. Source code will be released.

One-sentence Summary: This work explores a new paradigm for continual learning – learning to dynamically prompt the model to learn tasks sequentially under different task transitions.

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/arxiv:2112.08654/code)