US-SAM: An Automatic Prompt Sam For Ultrasound Image

Yuteng Wang, Xing Wu, Zhongshi He, Peng Wang, Haidong Wang, Hongqian Wang

Abstract: Segment anything model (SAM) has shown promising segmentation capabilities , however its performance significantly declines when applied to ultrasound images. Many works have emerged to address this issue, but still have two deficiencies: 1) SAM fine-tuning relies on manual prompts, which only allowing semiautomatics segmentation; 2) They exclusively utilize the frozen SAM encoder as the sole image encoder, which lacks pathological information. In this paper, we propose US-SAM which improve SAM with three modules: Pathological Extractor (PE), Fusion Module (FM), and Automatic Prompt Module (APM). PE extracts semantic information of lesions within the ultrasound image. Furthermore, FM fuses the features from PE and SAM image encoder. Finally, APM automatically generates prompts required by SAM using the fused features and uncertainty map. Through extensive experiments on two public ultrasound datasets BUSI and TN3k, our method outperforms other medical SAM methods by nearly 17% in Dice and IOU scores without any prompt from human.