Exploring Markov-chain-style Evidence Using LLMs for Complex Claims

Jinpeng Zhang, Ruifang He, Chang Liu, Bo Wang, Xiaohong Li, Junjie Wang

Keywords: Evidence Discovery, Large Language Model, Heuristic Search, Complex Claim

Abstract: The remarkable success of rationale generation provokes the precise evidence discovery, which aims to identify a small subset (evidence) from the context to infer a target claim. However, existing general methods often fall short in accurately modeling evidence strength and collaborative support. This paper reformulates evidence discovery as a multi-step prompt construction process and introduces a heuristic search framework, named McsE, to explore \textbf{M}arkov-\textbf{c}hain-\textbf{s}tyle \textbf{E}vidence. Specifically, we propose a novel strength modeling perspective: Large Language Models (LLMs) can effectively serve as reward functions to estimate evidence strength when appropriately prompted. Then, we incorporate independent and collaborative reward mechanisms to systematically explore diverse potential reasoning paths, ultimately establishing the most effective prompt path as evidence. Experiments conducted on three widely-used datasets show that the proposed framework outperforms seven baselines, with distinct advantages in extracting complex evidence.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 1018