Hierarchical Vision Transformers for Disease Progression Detection in Chest X-Ray Images

Amarachi Mbakawe, Lyuyang Wang, Mehdi Moradi, Ismini Lourentzou

Abstract: Chest radiography is a commonly used diagnostic imaging exam for monitoring disease progression and treatment effectiveness. While machine learning has made significant strides in tasks such as image segmentation, disease diagnosis, and automatic report generation, more intricate tasks such as disease progression monitoring remain fairly underexplored. This task presents a formidable challenge because of the complex and intricate nature of disease appearances on chest X-ray images, which makes distinguishing significant changes from irrelevant variations between images challenging. Motivated by these challenges, this work proposes CheXRelFormer, an end-to-end siamese Transformer disease progression model that takes a pair of images as input and detects whether the patient’s condition has improved, worsened, or remained unchanged. The model comprises two hierarchical Transformer encoders, a difference module that compares feature differences across images, and a final classification layer that predicts the change in the patient’s condition. Experimental results demonstrate that CheXRelFormer outperforms previous counterparts. Code is available at https://github.com/PLAN-Lab/CheXRelFormer.