A Model Merging Method
Keywords: Model Merging, Model Selection, model merging, staged responses.
Abstract: At the NeurIPS 2024 LLM-Merging competition, we successfully developed a simple and effective model merging approach that generates a versatile, generalist model, applicable to a wide range of scenarios. Specifically, this method is easy to implement, prevents significant conflicts among component models, and improves overall model accuracy to some extent. Additionally, it is memory-efficient, allowing for the combination of multiple foundational models, further optimizing resource utilization.
Submission Number: 6
Loading