RoentMod: A Synthetic Chest X-Ray Modification Model to Identify Image Interpretation Model Shortcuts

Lauren H. Cooke, Matthias Jung, Jan M. Brendel, Nora M. Kerkovits, Borek Foldyna, Michael T Lu, Vineet K Raghu

Keywords: Diffusion models, medical image modification, modeling disease presence

TL;DR: This work introduces RoentMod: a generative deep learning model to alter an existing chest x-ray with a text prompt describing pathology like cardiomegaly or pneumonia and finds that CXR interpretation models are sensitive to off-target pathology.

Abstract: Deep learning models can accurately identify pathology on chest x-ray (CXR) images in research settings but often have worse performance in external testing in part due to shortcut learning, where models rely on confounding factors in the training data rather than truly learning the appearance of target pathology. This work introduces RoentMod: a generative deep learning model to alter an existing CXR with a text prompt describing pathology like cardiomegaly or pneumonia. Using RoentMod, we find that CXR interpretation models are sensitive to the addition of off-target pathology, suggesting the use of shortcuts.

Submission Number: 62