Biomedical Question Answering via Multi-Level Summarization on a Local Knowledge Graph

ACL ARR 2025 May Submission4799 Authors

Abstract: In Question Answering (QA), Retrieval Augmented Generation (RAG) has revolutionized performance in various domains. However, how to effectively capture multi-document relationships remains an open question. This is particularly critical for biomedical tasks due to their reliance on information spread across multiple documents. In this work, we propose a novel method CLAIMS, which utilizes propositional claims to construct a local knowledge graph from retrieved documents. Summaries are then derived via layerwise summarization from the knowledge graph to contextualize a small language model to perform QA. The structured summaries effectively capture explicit and implicit relationships between entities in the documents, thus having a more comprehensive context to provide to LLMs. CLAIMS achieved comparable or superior performance over RAG baselines on several biomedical QA benchmarks. We also evaluated each individual step of our approach with a targeted set of metrics, demonstrating its effectiveness.

Paper Type: Long

Research Area: Question Answering

Research Area Keywords: Information Extraction, Information Retrieval and Text Mining, Question Answering, Summarization

Contribution Types: NLP engineering experiment

Languages Studied: English

Submission Number: 4799