HLAE: Hierarchical Local Attention Encoder for MRI Brain Tumor Image Classification

Changxu Dong, Dengdi Sun, Bin Luo

Abstract: MRI-based brain tumor classification is a challenging neuroimaging task, where the key lies in leveraging ensemble information from brain images. However, current algorithms primarily encode global appearance features of brain images and fail to account for local dependencies inherent in brain tissue adequately. In addition, most existing approaches do not thoroughly investigate the importance coefficients among different regions of brain images. To address these issues, in this work, we propose a novel cognitively-inspired hierarchical local attention encoder (HLAE) framework for capturing local dependency information in MRI images. Based on the characteristics of MRI images, we focus on local information from two perspectives: long-range visual feature dependencies and high-order structural context correlations to fully describe the content association and location relations of brain MRI images. For this purpose, a Swin Transformer is first utilized for encoding the patch-wise content dependencies of a brain MRI image by shifting windows and skipping connections. Meanwhile, a graph structure is also extracted from the MRI image, and a graph attention network is employed to capture the image’s contextual correlations. Finally, the two local information are integrated, and a softmax layer is used to obtain the final brain tumor classification result. This framework naturally contains the attention mechanism, which can effectively quantify the importance among different brain image regions, so as to locate the most discriminative regions in brain tumor classification accurately. Extensive experiments are conducted on two publicly available brain tumor MRI datasets. The results demonstrate its ability to automatically detect brain tumors with superior performance compared to state-of-the-art algorithms.