Automatic Correction of AI Reports using Fact-Checking Model-guided LLMs

Raziuddin Mahmood, Pingkun Yan, Tanveer F Syeda-Mahmood

Keywords: AI-generated report, fact-checking, ML models, LLM correction

TL;DR: Automatic Correction of AI Reports by using a fact-checking model to detect errors and defining instructional prompts to correct using LLMs

Abstract: Automated radiology report generators are being increasingly explored in clinical workflow pilots, particularly for chest X-ray imaging. However, retaining factual correctness with minimal hallucinations with respect to the description of the findings has often been lacking, making their adoption slow and requiring detailed verification by clinical experts. In this paper, we propose an automatic report correction method that uses both image and textual information in automated radiology reports to spot identity and location errors in findings through fact-checking models. Based on these errors, prompts are generated for selectively modifying the report sentences by a pre-trained LLMs. We show that this method of report correction, on the average, improves the report quality between 17-30\% across various SOTA report generators over multi-institutional chest X-ray datasets.

Submission Number: 104