Clinically-aligned Multi-modal Chest X-ray Classification

Phillip Sloan, Edwin Simpson, Majid Mirmehdi

Keywords: Medical Image Analysis, Medical Image Classification, Chest X-ray Classification

TL;DR: CaMCheX is a multimodal model that integrates multi-view chest X-rays with structured clinical data, achieving state-of-the-art performance on the MIMIC-CXR labels and the CXR-LT 2023 and 2024 benchmarks.

Track: Proceedings

Abstract: Radiology is essential to modern healthcare, yet rising demand and staffing shortages continue to pose major challenges. Recent advances in artificial intelligence have the potential to support radiologists and help address these challenges. Given its widespread use and clinical importance, chest X-ray classification is well suited to augment radiologists’ workflows. However, most existing approaches rely solely on single-view, image-level inputs, ignoring the structured clinical information and multi-image studies available at the time of reporting. In this work, we introduce CaMCheX, a multimodal transformer-based framework that aligns multi-view chest X-ray studies with structured clinical data to better reflect how clinicians make diagnostic decisions. Our architecture employs view-specific ConvNeXt encoders for frontal and lateral chest radiographs, whose features are fused with clinical indications, history, and vital signs using a transformer fusion module. This design enables the model to generate context-aware representations that mirror reasoning in clinical practice. Our results exceed the state of the art for both the original MIMIC-CXR dataset and the more recent CXR-LT benchmarks, highlighting the value of clinically grounded multimodal alignment for advancing chest X-ray classification.

General Area: Models and Methods

Specific Subject Areas: Medical Imaging

Data And Code Availability: Not Applicable

Ethics Board Approval: No

Entered Conflicts: I confirm the above

Anonymity: I confirm the above

Submission Number: 32