Enhancing LLM Fine-Tuning via Selective Parameter Merging

Anonymous

Abstract: Supervised fine-tuning (SFT) is a crucial technique for tailoring the generalization capacity of Large Language Models (LLMs) to specific target tasks. This study investigates enhancing LLMs fine-tuning through parameter merging technique. By merging models fine-tuned with varied data order, we achieve an enhanced SFT model demonstrating improved performance and lower validation losses. To our best knowledge, this is the first introduction of ``parameter-selection merging" technique, which innovatively merges models by selecting parameters from one sub-model in each parameter dimension, surpassing traditional weighted-average method across 5 datasets. Furthermore, this method has also shown superiority in multi-task merging scenarios, indicating a promising avenue for future LLM optimizations.

Paper Type: short

Research Area: NLP Applications

Contribution Types: NLP engineering experiment

Languages Studied: English