Learn to Merge: Meta-Learning for Adaptive Multi-Task Model Merging

Jun Chen, Qin Zhang, Weizhi Zhang, Xiao Luo, Philip S. Yu, Ziyue Qiao

Keywords: Model Merging, Meta Learning, Parameter Efficient Fine-Tuning

Abstract: Model merging in the pretrain-finetune paradigm has proven effective by combining multiple finetuned models into one with multi-task capabilities. Recent merging methods aim to boost merged models’ performance through strategies such as mitigating conflicts, adding trainable modules, and incorporating task-specific components. In most methods, the parameter merging procedure is based on Task Arithmetic, a widely used technique that creates task vectors from each finetuned model and linearly combines them with coefficients into consolidated model parameters. Except for studies specifically focusing on the merging coefficients, many other methods treat them as hand-tuned hyperparameters. However, the merging coefficients, which govern the entire merging process, including the subsequent module training, are empirically crucial for achieving optimal performance and tradeoff across tasks. Thus, this paper proposed an innovative model merging framework called MetaMerging, which constructs the merged model with a unified model and lightweight task-specific adapters. Specifically, the adapters are efficiently trained without labels via feature alignment with fine-tuned models, while the unified model is obtained by merging task vectors with coefficients adaptively optimized through meta-learning, which enhances the generalization and enables more effective adapter training. Extensive experiments on CV and NLP fields show strong performance of MetaMerging on various downstream tasks and demonstrate the effectiveness of meta-learning in our method compared to other parameter merging methods. Our code is available at https://anonymous.4open.science/r/MetaMerging-53A1

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 12573