Chain of Evidential Natural Language Inference: Advancing Biomedical Claim Verification powered by Large Language Models

ACL ARR 2024 December Submission399 Authors

Abstract: With the rise in biomedical research and the increasing risk of misinformation, ensuring the accuracy of claims about treatment effectiveness is critical, as inaccuracies can significantly affect patient care and treatment decisions. In this work, we introduce the Chain of Evidential Natural Language Inference(CoENLI) framework that leverages large language models (LLMs) to enhance natural language inference in biomedical claim verification. The task involves determining the entailment relationship between a claim and evidence derived from medical studies or clinical trial reports (CTRs). CoENLI enhances the ability of LLMs to process complex contexts and make logical inferences through a structured reasoning framework, comprising four clearly defined steps: \textit{semantic grounding, evidence-based evaluation, logical conclusion, and relation prediction}. Our experimental results demonstrate that, through structured, human-like deductive reasoning, small-scale LLMs can exhibit biomedical expertise and achieve high accuracy in biomedical claim verification.

Paper Type: Long

Research Area: NLP Applications

Research Area Keywords: Biomedical Claim Verification, Natural Language Inference, Large Language Models, Chain of Thought

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 399