IterKey: Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation

Kazuki Hayashi, Hidetaka Kamigaito, Shinya Kouda, Taro Watanabe

Keywords: retrieval-augmented generation, RAG, sparse retrieval, LLM, Iterative

TL;DR: We introduce IterKey, an LLM-based iterative keyword generation method that optimize the Retrieval-Augmented Generation process, improving accuracy by refining keywords and self-evaluating responses.

Abstract:

Retrieval Augmented Generation (RAG) has emerged as a way to complement the in-context knowledge of Large Language Models (LLMs) by integrating external documents.

However, real-world applications demand not only accuracy but also interpretability.
Dense retrieval methods provide high accuracy but lack interpretability, while sparse retrieval is transparent but often misses query intent due to keyword matching.
Thus, balancing accuracy and interpretability remains a challenge.

To address these issues, we introduce IterKey, an LLM-driven iterative keyword generation framework that enhances RAG via sparse retrieval.
IterKey consists of three LLM-driven stages: generating keywords for retrieval, generating answers based on retrieved documents, and validating the answers. If validation fails, the process iteratively repeats with refined keywords.

Across four QA tasks, experimental results show that IterKey achieves 5% to 20% accuracy improvements over BM25-based RAG and simple baselines. Its performance is comparable to dense retrieval based RAG and prior iterative query refinement methods using dense models.
In summary, IterKey is a novel BM25-based iterative RAG framework that leverages LLMs to balance accuracy and interpretability.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the COLM Code of Ethics on https://colmweb.org/CoE.html

Author Guide: I certify that this submission complies with the submission instructions as described on https://colmweb.org/AuthorGuide.html

Submission Number: 1129