Reliable Decision‑Making via Calibration‑Oriented Retrieval‑Augmented Generation

Chaeyun Jang, Deukhwan Cho, Seanie Lee, Hyungi Lee, Juho Lee

Keywords: Calibration, Uncertainty, LLMs, RAG

TL;DR: In this paper, we propose uncertainty calibration method, called CalibRAG, for the RAG guided decision-making scenario.

Abstract: Recently, Large Language Models (LLMs) have been increasingly used to support various decision-making tasks, assisting humans in making informed decisions. However, when LLMs confidently provide incorrect information, it can lead humans to make suboptimal decisions. To prevent LLMs from generating incorrect information on topics they are unsure of and to improve the accuracy of generated content, prior works have proposed Retrieval Augmented Generation (RAG), where external documents are referenced to generate responses. However, previous RAG methods focus only on retrieving documents most relevant to the input query, without specifically aiming to ensure that the human user's decisions are well-calibrated. To address this limitation, we propose a novel retrieval method called Calibrated Retrieval-Augmented Generation (CalibRAG), which ensures that decisions informed by RAG are well-calibrated. Then we empirically validate that CalibRAG improves calibration performance as well as accuracy, compared to other baselines across various datasets.

Primary Area: Deep learning (e.g., architectures, generative models, optimization for deep networks, foundation models, LLMs)

Submission Number: 16279