PiSSA-MoDCN: A Mixture-of-DCN for Segment Anything to Remote Sensing Ship for Semantic Segmentation

Zhiyong Xia, Wentian Cai, Ying Gao, Yandan Chen, Yang Ao, Zhengzhi Hou, Qin Qiu

Abstract: The Segment Anything Model (SAM), with its vast array of parameters and training data, possesses a rich semantic information and excellent generalization capabilities, performing remarkably across a variety of scenarios. This has inspired exploration into its potential applications in downstream tasks. In this paper, we propose PiSSA-MoDCN, a simple yet effective parameter-efficient fine-tuning method that aligns SAM with the field of remote sensing ship image processing. By employing Singular Value Decomposition (SVD) to initialize LoRA, it retains SAM's segmentation capabilities without introducing new parameters, thereby accelerating training speed. The designed MoDCN module leverages both Mixture of Experts (MoE) and deformable convolutional networks to enhance local image priors, infusing information from remote sensing ship images to improve model performance. Through extensive comprehensive experiments, compared to traditional advanced semantic segmentation algorithms and PEFT methods, PiSSA-MoDCN achieves higher performance with minimal computational resources, highlighting the potential of utilizing foundational models like SAM for specific downstream tasks in the field of remote sensing ship image processing.