Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Method

Anonymous

TL;DR: We propose a novel self-detection method to detect which questions an LLM does not know generating nonfactual responses.

Abstract:

Large Language Models (LLMs) have shown great potential in Natural Language Processing (NLP) tasks.
However, recent literature reveals that LLMs hallucinate intermittently, which impedes their reliability for further utilization.
In this paper, we propose a novel self-detection method to detect which questions an LLM does not know.
Our proposal is empirical and applicable for continually upgrading LLMs compared with state-of-the-art methods.
Specifically, we examine the divergence of the LLM's behaviors on different verbalizations for a question and examine the atypicality of the verbalized input.
We combine the two components to identify whether the model generates a non-factual response to the question.
The above components can be accomplished by utilizing the LLM itself without referring to any other external resources.
We conduct comprehensive experiments and demonstrate the effectiveness of our method for recently released LLMs involving Llama 2, Vicuna, ChatGPT, and GPT-4 across factoid question-answering, arithmetic reasoning, and commonsense reasoning tasks.

Paper Type: long

Research Area: Interpretability and Analysis of Models for NLP

Contribution Types: Model analysis & interpretability

Languages Studied: English