Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach

Anonymous

TL;DR: This paper proposes a numerical method to identify hallucinations in LLM content, achieving promising outcomes in Summarization and Question Answering by evaluating token probabilities and offering an efficient real-time evaluation.

Abstract: With the rise of Large Language Models (LLMs) in recent times, concerns about their tendency to hallucinate and produce inaccurate outputs have also increased. Detecting such hallucinations is crucial for ensuring trustworthiness in applications relying on LLM-generated content. Current methods, often resource-intensive and reliant on extensive LLMs and intricate linguistic and semantic analyses, are not easily reproduced. This paper seeks to introduce a simpler method to detect hallucinations in LLM generations using purely numerical features. By evaluating token probabilities within the generated content and vocabulary, the method achieves promising results, surpassing state-of-the-art outcomes in Summarization and Question Answering on the Hallucination Evaluation for Large Language Models (HaluEval) benchmark. This method demonstrates effectiveness in pinpointing hallucinatory content, offering a more efficient pathway for real-time LLM output evaluation without the need for intricate linguistic analyses.

Paper Type: long

Research Area: Machine Learning for NLP

Contribution Types: NLP engineering experiment

Languages Studied: english