EDU-RAG: A RAG Benchmark with Web-enhanced Content in Education Domain. Can RAG Help AI Tutor?
Keywords: Large Language Models (LLM), Retrieval Augmented Generation (RAG), Search
Abstract: Hallucination has been a persistent challenge when using Large Language Models (LLMs). Retrieval-Augmented Generation (RAG) has emerged as a popular approach to mitigate this issue by maintaining context and coherence in generated outputs, as well as incorporating customized knowledge. In this paper, we propose a benchmark dataset for evaluating LLM performance in the domain of middle-school science question answering, using textbook questions augmented with real-world web search results. We assess the performance of various LLMs, including GPT-4o, Llama2-7b, and Llama3-8b, with and without the application of RAG. Our goal is to determine whether RAG can reduce hallucinations stemming from the inherent biases of pre-trained LLMs or from the retrieval of irrelevant knowledge, even when relevant information is accessible. The dataset and methodology introduced here provide a robust foundation for advancing the evaluation and development of RAG techniques in mitigating hallucinations across diverse LLMs.
Primary Area: datasets and benchmarks
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 13263
Loading