Sema-ChestX-Former: A Parameter-Efficient Hybrid Transformer-CNN for Robust Thoracic Disease Classification with XAI

Touhid Alam, Sayedur Rahman Anik, Nafiz Fahad, Md Tanzib Hosain, Md Kishor Morol, Md. Jakir Hossen

Keywords: x-ray, clinical imaging, radiology

Abstract: Thoracic diseases pose a significant global health challenge, and while deep learning has shown promise in analyzing chest X-rays (CXRs), current models face a critical trade-off. Convolutional Neural Networks (CNNs) excel at local feature extraction but miss global context, while Vision Transformers (ViTs) capture long-range dependencies but are often parameter-heavy and computationally expensive. This study proposes the Semantic Chest X-ray Transformer (Sema-ChestX-Former), a novel, parameter-efficient (~1.84 million parameters) hybrid architecture designed to overcome these limitations. Our model synergistically integrates a Transformer backbone for semantic spatial feature extraction with specialized CNN-based attention blocks for fine-grained feature refinement. We conducted a comprehensive evaluation on three large-scale public datasets—Chest X-Ray (Pneumonia), COVID-19 Radiography, and NIH ChestX-ray14—covering binary, multi-class, and multi-label classification challenges. Furthermore, we employed Explainable AI (XAI) using Gradient-weighted Class Activation Mapping (Grad-CAM) to ensure model transparency and interpretability. Sema-ChestX-Former established comparative performance on the NIH ChestX-ray14 dataset with a mean AUC of 0.846, while also achieving 99.69\% accuracy on the Pneumonia dataset and 98.34\% on the COVID-19 dataset. Our findings demonstrate that a carefully designed, parameter-efficient hybrid architecture can outperform larger, more complex models, offering a promising and practical solution for automated CXR analysis in clinical settings.

Primary Subject Area: Integration of Imaging and Clinical Data

Secondary Subject Area: Application: Radiology

Registration Requirement: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 319