Adapting Segment Anything 2 for Diabetic Retinopathy Lesion Segmentation

Ying Chen, Xinyu Xiong, Huihui Fang, Yanwu Xu

Abstract: Automated segmentation of diabetic retinopathy (DR) lesions in fundus images plays a crucial role in computer-aided diag-nosis. While existing research has explored lesion segmentation extensively, many approaches rely on traditional, smaller backbone networks for feature extraction. In contrast, foun-dation segmentation models, benefiting from pretraining on vast, high-quality segmentation datasets, offer more efficient feature extraction capabilities. Therefore, this paper proposes to adapt the Segment Anything Model 2 (SAM2) to enhance DR lesion segmentation. Specifically, we freeze the parameters of the SAM2 Hiera encoder and leverage adapter modules for efficient fine-tuning. The decoder employs a multi-scale block design, mirroring the encoder's architecture, to achieve efficient segmentation. Experimental validation on the publicly available IDRiD and DDR datasets demonstrates the effectiveness of our proposed network.