Mitigating Hallucination in Large Language Models with Explanatory Prompting

Alexander Braverman, Weitong Zhang, Quanquan Gu

Abstract:

A growing concern with the use of Large Language Models (LLMs) is the presence
of hallucinated outputs. For tasks that require complex reasoning, hallucinations
make LLMs unreliable and thus unsafe to deploy in a range of applications from
healthcare to education. To combat this issue, we propose explanatory prompting,
a methodology that gives an informal logical description of an algorithm needed
to solve all instances of a given problem. To illustrate the use of explanatory
prompting, we consider a Graph Connectivity problem on directed acyclic graphs.
We evaluate our approach by experiments on the Flight Connectivity dataset, an
instance of a Graph Connectivity problem (Zhang et al., 2023a). Our experiments
demonstrate a decrease in hallucination rate from 44.8% in prior work to 1.8%
using explanatory prompting. At the same time, we confirm that calibrated LLMs
are bound to hallucinate by experimentally verifying a theoretical lower bound for
hallucination (Kalai and Vempala, 2024).