Two heads are better than one: Enhancing LLMs Reasoning with Model Ensemble

Anonymous

Abstract: Each Large Language Model (LLM) possesses unique strengths and limitations, urging the model ensemble to take full advantage of complementary strengths of different LLMs. To achieve this, we propose novel model ensemble methods which combine the confidence and popularity scores to generate the final outputs. The confidence is measured by the belief degree of one LLM to produce its output and the popularity is evaluated through the consistency degree of its output to other LLMs. Experimental results demonstrate that our methods markedly improve the performance on seven commonly used reasoning benchmarks, surpassing both the top-performing model and other strong baselines. Additionally, we explore the effects of varying ensemble sizes, offering valuable insights for optimizing model ensemble strategies for LLMs reasoning.

Paper Type: long

Research Area: NLP Applications

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English