TKG-RAG: A Retrieval-Augmented Generation Framework with Text-chunk Knowledge Graph

Wei Xiao, Yu Liu, XiangLong Li, Feng Gao, Jinguang Gu

Abstract: The approach of dividing text into chunks and building indexes is a mainstream approach for Retrieval-Augmented Generation (RAG) of Large Language Models (LLMs). However, retrieved text chunks often contain noise or redundant information, which can negatively impact RAG performance. To address this limitation, researchers have proposed RAG approaches based on text chunks, knowledge graphs, and long-texts. Nevertheless, there are still challenges that need to be addressed, such as the underutilization of knowledge within text chunks, the resource-intensive nature and complex process of constructing Knowledge Graphs (KG), and the lack of emphasis on optimization during the post-processing filtering stage. We propose a RAG framework with text-chunk knowledge graph (TKG-RAG). The proposed framework can construct a text-chunk knowledge graph automatically by extracting the hierarchical structure, contextual relationships, topic sentences, and inter-chunk relationships from the domain text. The framework begins by using text indexing to retrieve relevant text chunks based on similarity. These chunks are then mapped to nodes within the text-chunk knowledge graph, where connections are established based on the relationships of nodes in the graph to generate subgraphs. The content of the nodes is rearranged and merged using the hierarchical structure and relational knowledge in the graph, and then the residual isolated nodes that are not merged are fused based on the attribute knowledge in the TKG. In addition, to improve the performance of filters in the post-processing stage, we incorporate the datasets considering texts and numerical characteristics, into the fine-tuning process of the filter model. This approach aimed to enhance the accuracy of filtering and reduce token consumption in the generation stage. To validate the effectiveness of this approach, comparative and ablation experiments were conducted on five datasets: NQ, PopQA, HotpotQA, TriviaQA, and LawQA. The results show that TKG-RAG can achieve better performance in terms of Accuracy and F1 scores, while also reducing token consumption by 46%, means that TKG-RAG can combine the strengths of chunk-based and graph-based RAG approaches.