Small Empowering Large: Leverage Performance and Efficiency by Exploring the Cooperation of LLMs

Anonymous

Abstract: The combined use of Large Language Models (LLMs) and Information Retrieval (IR) has made significant progress in solving the multi-hop QA problem. However, achieving high performance requires increasingly complex and interactive integration of IR and 'large' LLMs, which poses challenges to efficiency and domain specialization capabilities. A specifically fine-tuned 'small' LLM, such as LlaMa-7B, presents a viable solution to this challenge.Nevertheless, addressing the challenges entails considering two aspects: 1) Where Problem: identifying the phases in which employing a 'small' LLMs is most beneficial is essential. 2) How Problem: devising effective strategies for combining 'small' and 'large' LLMs is necessary.A lightweight approach is proposed where the 'large' LLMs service and a specifically fine-tuned 'small' LLMs cooperate to answer the multi-hop questions. Our research reveals that the 'large' LLMs service primarily handles top-level planning, while the fine-tuned 'small' LLMs is tasked with generating answers and rectifying any inconsistencies with the retrieved information. Experimental results on the HotPotQA dataset demonstrate that our proposed method achieves comparable accuracies with significantly reduced costs.

Paper Type: long

Research Area: Question Answering

Contribution Types: NLP engineering experiment, Approaches to low-resource settings

Languages Studied: English