Efficient Model Development through Recycling Fine-tuning

ACL ARR 2025 February Submission5637 Authors

Abstract: Modern large language models (LLMs) undergo a two-stage development process: pretraining on large corpora followed by post-training, which includes fine-tuning and reinforcement learning to align with human preferences and improve performance on specific downstream tasks. However, when a new model version is released, the fine-tuning process needs to be repeated, incurring significant computational and resource costs. In this work, we propose a method to recycle fine-tuning across model versions by transferring weight changes, or diff vectors, from a previous fine-tuned model to a new base model. We empirically validate this approach across multiple open-weight models, demonstrating that the transferred diff vectors can significantly enhance the new model’s performance, often achieving results competitive with direct fine-tuning. Through controlled experiments, we establish that fine-tuning transfer is most effective when the source and target models are linearly connected in the parameter space. Furthermore, we explore applications in multilingual model development, showing that recycling fine-tuning improves performance on target-language tasks without additional training. Finally, we introduce an iterative merging approach for continuous model development, which further enhances efficiency and effectiveness. Our findings suggest that fine-tuning recycling is a viable strategy to reduce training costs while maintaining model performance.

Paper Type: Long

Research Area: Efficient/Low-Resource Methods for NLP

Research Area Keywords: recycling finetuning, model merging, model development, diff vector, efficient transfer

Contribution Types: Approaches to low-resource settings, Approaches low compute settings-efficiency

Languages Studied: English,Malagasy,Shihala,Turkish

Submission Number: 5637