Learning to Recover Task Experts from a Multi-Task Merged Model

Jinwook Jung, Yeoreum Lee, Kumju Jo, Taegyu Kim, Sungyong Baik

Keywords: Model merging, dynamic model merging

Abstract: Multi-task model merging aims to merge several task-specific models (or experts) into a unified multi-task model. However, model merging often results in performance degradation due to parameter interference between experts. While several recent works have focused on improving the merging process to mitigate the parameter interference, there still exists the performance gap between merged models and task experts. In this work, we take a different perspective: we aim to recover a task expert from a merged model, instead of trying to improve the merging process. We first note that the parameter interference arises, as a merging process introduces offsets to expert model parameters. Thus, we propose to learn to **Re**cover a **T**ask **eX**pert (**ReTeX**) model, by undoing this offset. Specifically, we train a lightweight linear module to predict the offset to recover a task expert for a given input. Experiments demonstrate that ReTeX significantly outperforms existing model merging methods across computer vision domains and NLP domains with models of various scales, recovering more than 99% of individual expert performance even when scaling to 30 tasks. Furthermore, ReTeX can be applied to several existing merging models, demonstrating its flexibility and applicability.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 4297