Inter-Scale Dependency Modeling for Skin Lesion Segmentation with Transformer-based Networks

Sania Eskandari, Janet Lumpp

Keywords: Deep learning, Transformer, Skin lesion segmentation, Inter-scale context fusion

Abstract: Melanoma is a dangerous form of skin cancer caused by the abnormal growth of skin cells. Early diagnosis is critical to effective treatment, and several Fully Convolutional Network (FCN) approaches, including the U-Net architecture, can automatically segment skin lesions to aid diagnosis. The symmetrical U-Net model has shown outstanding results, but its use of a convolutional operation limits its ability to capture long-range dependencies, which are essential for accurate medical image segmentation. In addition, the U-shaped structure suffers from the semantic gaps between the encoder and decoder. In this study, we developed and evaluated a U-shaped hierarchical Transformer-based structure for skin lesion segmentation while we proposed an Inter-scale Context Fusion (ISCF) to utilize the attention correlations in each stage of the encoder to adaptively combine the contexts coming from each stage to hinder the semantic gaps. The preliminary results of the skin lesion segmentation benchmark endorse the applicability and efficacy of the ISCF module.