A Novel Clinical Trial Prediction-Based Factual Inconsistency Detection Approach for Medical Text Summarization

Shuaimin Li, Jungang Xu

Abstract: Most existing works of factual inconsistency detection focus on text summarization of generic articles. In this paper, a clinical trial prediction-based factual inconsistency detection approach is proposed for medical text summarization. Inspired by the fact that medical articles related to a clinical trial can give some evidence to predict the outcome of this trial, we believe that a factual consistent summary of the medical articles can also accurately predict the outcome of the corresponding clinical trial. Therefore, we first gather a novel Clinical Trial Prediction-based summarization (CTPSum) dataset, which is a collection of the outcomes of the clinical trials together with the related medical articles and summaries. We then propose a keyword-aware classification model to predict the outcome (successful/failed) of a clinical trial. If the predicted outcome is correct, the generated summary is considered to be factually consistent with the source document, otherwise, it is labeled as factually inconsistent. To evaluate the proposed approach, we further collect a Fact Inconsistency Detection dataset in the Medical Domain (FIDMD), which includes summaries, medical articles, and binary labels indicating factual consistency or inconsistency. Experimental results demonstrate the effectiveness of the proposed approach, specifically, the clinical trial prediction-based factual inconsistency detection approach outperforms several NLI-based factual inconsistency detection methods on the FIDMD dataset.