Joint Region-Attention and Multi-scale Transformer for Microsatellite Instability Detection from Whole Slide Images in Gastrointestinal Cancer

Zhilong Lv, Rui Yan, Yuexiao Lin, Ying Wang, Fa Zhang

Abstract: Microsatellite instability (MSI) is a crucial biomarker to clinical immunotherapy in gastrointestinal cancer, while additional immunohistochemical or genetic tests for MSI are generally missing due to lack of medical resources. Deep learning has achieved promising performance in detecting MSI from hematoxylin and eosin (H &E) stained histopathology slides. However, these methods are primarily based on patch-supervised slide-label models and then aggregate patch-level results into the slides-level result, resulting unstable prediction due to noisy patches and aggregation ways. In this paper, we propose a joint region-attention and multi-scale transformer (RAMST) network for microsatellite instability detection from whole slide images in gastrointestinal cancer. Specifically, we present a region-attention mechanism and a feature weight uniform sampling (FWUS) method to learn a representative subset of image patches from whole slide images. Moreover, we introduce the transformer architecture to fuse the multi-scale histopathology features consisting of patch-level features with region-level features to characterize the whole slide images for slide-level MSI detection. Compared to the existing MSI detection methods, the proposed RAMST shows the best performances on the colorectal and stomach cancer dataset from The Cancer Genome Atlas (TCGA) and provides an effective features representation learning method for WSI-label tasks.