CheXagent: Towards a Foundation Model for Chest X-Ray Interpretation

Zhihong Chen, Maya Varma, Jean-Benoit Delbrouck, Magdalini Paschali, Louis Blankemeier, Dave Van Veen, Jeya Maria Jose Valanarasu, Alaa Youssef, Joseph Paul Cohen, Eduardo Pontes Reis, Emily Tsai, Andrew Johnston, Cameron Olsen, Tanishq Mathew Abraham, Sergios Gatidis, Akshay S Chaudhari, Curtis Langlotz

Track: Non-traditional track

Keywords: AI in health, Foundation Models, Chest X-rays

TL;DR: We design a complete scheme for developing CXR FMs by introducing CheXisntruct, CheXagent, and CheXbench, where experimental results demonstrate its effectiveness.

Abstract: Chest X-rays (CXRs) are the most frequently performed imaging test in clinical practice. Recent advances in the development of vision-language foundation models (FMs) give rise to the possibility of performing automated CXR interpretation. In this work, we present (i) \emph{CheXinstruct} - a large-scale instruction-tuning dataset curated from 28 publicly-available datasets; (ii) \emph{CheXagent} - an instruction-tuned FM capable of analyzing and summarizing CXRs; and (iii) \emph{CheXbench} - a novel benchmark designed to systematically evaluate FMs across 8 clinically-relevant CXR interpretation tasks. Extensive quantitative evaluations and qualitative reviews with five expert radiologists demonstrate that CheXagent outperforms previously-developed general- and medical-domain FMs on CheXbench tasks by up to 97.5\%.

Presentation And Attendance Policy: I have read and agree with the symposium's policy on behalf of myself and my co-authors.

Ethics Board Approval: No, our research does not involve datasets that need IRB approval or its equivalent.

Data And Code Availability: Yes, we will make data and code available upon acceptance.

Primary Area: Clinical foundation models

Student First Author: Yes, the primary author of the manuscript is a student.

Submission Number: 27