Effective and Parameter-Efficient Reusing Fine-Tuned Models

Weisen Jiang, Baijiong Lin, Han Shi, Yu Zhang, Zhenguo Li, James Kwok

Keywords: merging models, fine-tune models

Abstract: Many pre-trained large-scale models provided online have become highly effective in transferring to downstream tasks. At the same time, various task-specific models fine-tuned on these pre-trained models are available online for public use. In practice, collecting task-specific data is labor-intensive and fine-tuning the large pre-trained models is computationally expensive, one can reuse task-specific fine-tuned models to deal with downstream tasks. However, using a model per task causes a heavy burden on storage and serving. Recently, many training-free and parameter-efficient methods have been proposed for merging multiple fine-tuned task-specific models into a single multi-task model. However, these methods exhibit a large accuracy gap compared with using a fine-tuned model per task. In this paper, we propose parameter-efficient methods for Reusing fine-tuned models. For reusing fully fine-tuned models, we inject sparse task vectors to a merged model by magnitude pruning. For reusing LoRA fine-tuned models, we use a lower-rank matrix to approximate the LoRA matrix by singular value decomposition. Extensive experiments conducted on computer vision and natural language process tasks demonstrate the effectiveness and parameter-efficiency of the proposed methods. The proposed methods outperform existing merging models method by a large margin and achieve comparable performance to using a fine-tuned model per task.

Primary Area: general machine learning (i.e., none of the above)

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7115