What External Knowledge is Preferred by LLMs? Characterizing and Exploring Chain of Evidence in Imperfect Context

ACL ARR 2025 February Submission4785 Authors

Abstract:

Incorporating external knowledge into large language models (LLMs) has emerged as a promising approach to mitigate outdated knowledge and hallucination in LLMs.
However, external knowledge is often imperfect.
In addition to useful knowledge, external knowledge is rich in irrelevant or misinformation in the context that can impair the reliability of LLM responses.
This paper focuses on LLMs' preferred external knowledge in imperfect contexts when handling multi-hop QA.
Inspired by criminal procedural law's Chain of Evidence (CoE), we characterize that knowledge preferred by LLMs should maintain both relevance to the question and mutual support among knowledge pieces.
Accordingly, we propose an automated CoE discrimination approach and evaluate LLMs' effectiveness, faithfulness and robustness with CoE, including its application in the knowledge-intensive task. Tests on five LLMs show CoE improves generation accuracy, answer faithfulness, robustness to knowledge conflicts, and performance in a knowledge-intensive task.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: counterfactual/contrastive explanations, feature attribution

Contribution Types: Model analysis & interpretability, Data analysis

Languages Studied: English

Submission Number: 4785