Can We Verify Step by Step for Incorrect Answer Detection?
Abstract: Chain-of-Thought (CoT) prompting has marked a significant advancement in enhancing the reasoning capabilities of large language models (LLMs).
Previous studies have developed various extensions of CoT, which focus primarily on enhancing end-task performance.
In addition, there has been research on assessing the quality of reasoning chains in CoT.
This raises an intriguing question: Is it possible to predict the accuracy of LLM outputs by scrutinizing the reasoning chains they generate?
To answer this research question, we introduce a benchmark, R2PE, designed specifically to explore the relationship between reasoning chains and performance in various reasoning tasks spanning five different domains.
This benchmark aims to measure the falsehood of the final output of LLMs based on the reasoning steps.
To make full use of information in multiple reasoning chains, we propose the process discernibility score (PDS) framework that beats the answer-checking baseline by a large margin.
Concretely, this resulted in an average of $5.1\%$ increase in the F1 score and $2.97\%$ improvement in AUC-PR across all 45 subsets within R2PE.
We further demonstrate our PDS's efficacy in advancing open-domain QA accuracy.
Paper Type: Long
Research Area: Interpretability and Analysis of Models for NLP
Research Area Keywords: Large Language Models, Chain of Thought, answer veracity, benchmark
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Reproduction study, Publicly available software and/or pre-trained models, Data resources
Languages Studied: English
Submission Number: 2683
Loading