Exploiting Presentative Feature Distributions for Parameter-Efficient Continual Learning of Large Language Models

Xin Cheng, Jiabo Ye, Haiyang Xu, Ming Yan, Ji Zhang, Feng Liu, Fei Huang, Lei Feng

TL;DR: We propose a novel CL method for LLMs that avoids a critical limitation called information leakage while maintaining outstanding CL performance.

Abstract: Endowing large language models (LLMs) with continual learning (CL) capacities is practically important, which enables them to dynamically acquire new knowledge over time. Although many effective methods have been proposed for CL of LLMs, they did not consider online scenarios, thereby sharing a common problem: information leakage (IL), where the task-related information of learned tasks is accessed or reused again. IL not only imposes potential risks on data privacy protection but also significantly hinders the deployment of LLMs in real-world scenarios. To avoid IL while maintaining outstanding CL performance, we propose a novel CL method for LLMs, which first characterizes a parameter-efficient fine-tuning (PEFT) block by a presentative feature distribution, and then dynamically selects the appropriate PEFT blocks for each instance based on its similarity with the presentative feature distributions. Extensive experiments validate the effectiveness of our method on the CL of LLM, showcasing its potential to enhance both privacy and adaptability in practical applications.

Lay Summary:

Providing large language models (LLMs) with continual learning abilities is crucial for them to dynamically acquire new knowledge over time. We analysed current advanced continual learning methods for LLMs in online scenarios and found that they either perform poorly or have the information leakage problem. We aim to develop a method that avoids information leakage while maintaining strong continual learning performance.

Our method leveraged well-developed pre-trained LLMs to describe the distribution of each streaming task. We discovered that by evaluating the distance between samples and distributions, we can effectively achieve good continual learning performance. Additionally, our method can also flexibly expand the knowledge trained with the same model framework, achieving plug-in continual learning.

Our in-depth study of information leakage in online continual learning can help better deploy LLMs with continual learning capabilities in real-world scenarios.

Link To Code: https://github.com/ZERO-9215/Online-CL-LLMs

Primary Area: Deep Learning->Large Language Models

Keywords: large language model, continual learning, low-rank adaptation

Submission Number: 10369