Bridging Knowledge Gaps: Fine-Tuned RAG Frameworks for Biomedical Evidence-Based Question Answering

Xiang-Yun Wang, Zhu-Hong You, Yu Li, Yun-Hui Yan, Yue-Chao Li

Abstract: Biomedical Question Answering (QA) systems face challenges due to the rapid evolution of domain-specific knowledge and the limitations of general-purpose language models. With the rapid development of fine-tuning techniques for large language models, their adaptability in the biomedical domain has significantly improved. Additionally, the emergence of Retrieval-Augmented Generation (RAG) technology enables large language models to retrieve external biomedical knowledge, allowing them to provide more accurate answers to domain-specific questions. Existing large language models, trained through extensive pre-training or full-parameter fine-tuning, often lack feasibility with limited data and resources. This paper proposes a method for parameter-efficient fine-tuning of the RAG generator, achieving results comparable to existing biomedical large models while using fewer data and resources. Specifically, our approach combines the training sets of MedMCQA and PubMedQA, fine-tunes the large language model using the Low-Rank Adaptation (LoRA) method, and integrates a biomedical domain vector database to construct a fine-tuned RAG framework. With limited data and computational resources, our method achieves performance on par with or even surpassing existing open-source or proprietary large models. We achieve an accuracy of 80.6% on PubMedQA, 58.52% on MedMCQA, and 91.91% on BioASQ-Y/N.

External IDs:dblp:conf/icic/WangYLYL25