D-RAG: Differentiable Retrieval-Augmented Generation for Knowledge Graph Question Answering

ACL ARR 2025 February Submission4727 Authors

Abstract:

Knowledge Graph Question Answering (KGQA) aims to answer natural language questions based on knowledge graphs.
Recent approaches apply the Retrieval-Augmented Generation (RAG) paradigm to incorporate Large Language Models (LLMs) to this task, where a retriever selects a question-related subgraph and an LLM-based generator is then adopted to predict answers based on the retrieved subgraph.
However, the subgraph selection process is non-differentiable, preventing end-to-end training of the retriever and the generator in these approaches, which leads to sub-optimal performance.
To overcome this limitation, this paper proposes a Differentiable RAG (D-RAG) approach that jointly optimizes the retriever and the generator for KGQA.
Firstly, D-RAG reformulates the optimization objective as an expectation over a subgraph distribution with respect to answer generation likelihood, making the joint optimization feasible.
Secondly, it designs a differentiable subgraph sampling and prompting module based on Gumbel-Softmax reparameterization, which achieves end-to-end optimization and allows the retriever to discover latent graph patterns that actively facilitate the generator's reasoning process.
Experimental results on WebQSP and CWQ show that D-RAG outperforms the state-of-the-art approaches by 2.3\% and 3.4\% on the F1 scores, respectively, demonstrating its effectiveness.

Paper Type: Long

Research Area: Question Answering

Research Area Keywords: knowledge base QA, knowledge graphs

Languages Studied: English

Submission Number: 4727