ToT-RAG: Improving Re-ranking of Retrieval-Augmented Generation with Tree of Thoughts

ACL ARR 2025 May Submission6109 Authors

Abstract: The Retrieval Augmented Generation (RAG) technique that utilize external knowledge can enable large language models to reduce illusions and perform well in numerous open-domain question answering (ODQA) tasks. The results of re-ranking, as a part of RAG, will be directly used in prompt of the large language model's input, which has a significant impact on the results of RAG system. Therefore, this paper proposes a method of re-ranking based on tree of thoughts (ToT) in RAG, to ensure the overall quality of the text retrieved. This paper not only proposes for the first time to re-rank the texts from multiple dimensions in RAG system, but also combines the large language models with agent to evaluate the text using the tree structure, so that the text obtained from re-ranking would have both outstanding text quality and a high degree of similarity with the user's input. ODQA experiments on three datasets demonstrate that ToT-RAG can effectively reduce illusions and improve the answer accuracy of the RAG system. In comparison experiments, we further illustrate that tree-structured re-ranking is optimal under the trade-off between resource consumption and task accuracy.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: NLP Applications

Languages Studied: English

Submission Number: 6109