NoLLMRAG: LLM-Free Makes Graph-Based RAG Highly Efficient, Effective and Generalizable

ChengZhuo Yu, Fang Su

Keywords: Retrieval-Augmented Generation, Large Language Model, Graph

TL;DR: We propose NoLLMRAG, a novel graph-based RAG framework which is LLM-free during indexing and retrieval, enabling strong performance, exceptional efficiency, and robust generalization

Abstract: Graph-based Retrieval-Augmented Generation (graph-based RAG) improves retrieval relevance and multi-hop reasoning compared to traditional RAG by constructing a graph that models relationships among text chunks. However, existing methods heavily rely on LLMs during indexing, resulting in inefficiency and unstable performance across LLM scales. Moreover, during retrieval, the lack of effective mechanisms for extracting query keywords and filtering irrelevant chunks further leads to redundant retrieval, introducing noise and degrading answer quality. To address these limitations, we propose NoLLMRAG, a novel graph-based RAG framework which is LLM-free during indexing and retrieval. It builds a three-layer heterogeneous graph index without LLMs, leverages a graph-statistics-driven keyword extraction to select keywords from queries that are aligned with the corpus, and applies a clustering-based retrieval on co-occurrence subgraphs to select more relevant chunks for generation. Experiments on three datasets and three LLMs demonstrate that NoLLMRAG achieves an average improvement of 41.27\% over the strongest baseline, with indexing speedup of up to 300$\times$ and QA speedup of up to 15$\times$, and maintains robust adaptability for real-time corpus expansion, highlighting its superior performance, efficiency, and generalization across LLMs and domains.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 5835