Fusing Highly Specialized Language Models for Comprehensive Expertise
Abstract: Underlying data distributions of natural language, programming code, and mathematical symbols vary vastly, presenting a complex challenge for large language models (LLMs) that strive to achieve high performance across all three domains simultaneously.
Achieving a very high level of proficiency for an LLM within a specific domain often requires extensive training with relevant corpora, which is typically accompanied by a sacrifice in performance in other domains.
In this paper, we aim to ``play the dealt cards well'' and propose to fuse models that are already highly-specialized directly.
The proposed fusing framework, \modelname, consists of different distinct specialists that are already sufficiently trained on different domains (we mainly focus on language, coding, and mathematics in this paper). A token-level gating mechanism is introduced to blend the specialists' outputs. A two-stage training strategy accompanied by balanced sampling is designed to ensure stability.
To effectively train the fused model, we further construct a high-quality supervised instruction tuning dataset, \dataname, which includes text, code, and mathematical content. This dataset comprises approximately 300,000 instructions and covers a wide range of topics in each domain. Experiments show that our model could simultaneously achieve mastery of the three crucial domains.
Paper Type: Long
Research Area: NLP Applications
Research Area Keywords: Language Models, Model Fusing
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data resources
Languages Studied: English
Submission Number: 232
Loading