KGMistral: Towards Boosting the Performance of Large Language Models for Question Answering with Knowledge Graph Integration

Mingze Li, Haoran Yang, Zhaotai Liu, Mirza Mohtashim Alam, Ebrahim, Harald Sack, Genet Asefa Gesese

Keywords: Knowledge Graph, Large Language Models, Question Answering, Retrieval Augmented Generation, SPARQL

Abstract: In this paper, a novel question-answering (QA) approach named KGMistral is proposed, based on the Retrieval Augmented Generation (RAG) framework. Given the limitations of Large Language Models (LLMs) in generating accurate answers for domains not adequately covered by their training corpus, this work focuses on leveraging external domain-specific Knowledge Graphs (KGs) to enhance the performance of LLMs. Specifically, the study examines the benefits of using information from a KG to improve the QA performance of the Mistral model in the material science and engineering field. Experimental results indicate that KGMistral significantly enhances Mistral’s QA performance.

Submission Number: 7