Large Continual Instruction Assistant

Jingyang Qiao, zhizhong zhang, Xin Tan, Yanyun Qu, Shouhong Ding, Yuan Xie

TL;DR: A general continual instruction tuning framework

Abstract: Continual Instruction Tuning (CIT) is adopted to continually instruct Large Models to follow human intent data by data. It is observed that existing gradient update would heavily destroy the performance on previous datasets during CIT process. Instead, Exponential Moving Average (EMA), owns the ability to trace previous parameters, which can aid in decreasing forgetting. Nonetheless, its stable balance weight fails to deal with the ever-changing datasets, leading to the out-of-balance between plasticity and stability. In this paper, we propose a general continual instruction tuning framework to address the challenge. Starting from the trade-off prerequisite and EMA update, we propose the plasticity and stability ideal condition. Based on Taylor expansion in the loss function, we find the optimal balance weight can be automatically determined by the gradients and learned parameters. Therefore, we propose a stable-plasticity balanced coefficient to avoid knowledge interference. Based on the semantic similarity of the instructions, we can determine whether to retrain or expand the training parameters and allocate the most suitable parameters for the testing instances. Extensive experiments across multiple continual instruction tuning benchmarks demonstrate that our approach not only enhances anti-forgetting capabilities but also significantly improves overall continual tuning performance. Our code is available at https://github.com/JingyangQiao/CoIN.

Lay Summary:

We want Large Foundation Models to continually learn from new human instructions without forgetting what they have learned before, which is called Continual Instruction Tuning. However, current training paradigms often make the model forget old knowledge when learning new tasks.

We improve this by proposing a novel technique called Large Continual Instruction Assistant, which helps the model remember past knowledge. By analyzing the trade-off between learning new information and avoiding forgetting, we derive a principled way to compute an optimal balance coefficient based on gradients and learned parameters. This allows the model to automatically adjust its updates based on what it is currently learning and what it has already learned.

We also measure how similar new instructions are to past ones, so the model can decide whether to reuse what it already knows or learn something new. In conclusion, our method successfully helps Large Foundation Models mitigate forgetting during learning new knowledge.

Link To Code: https://github.com/JingyangQiao/CoIN

Primary Area: General Machine Learning->Everything Else

Keywords: Catastrophic Forgetting, Continual Learning, Instruction Tuning, Exponential Moving Average, Large Foundation Models

Submission Number: 3738