GA-SAM: Geometry-Aware SAM Adaptation with Sparse Annotation-Driven Point Cloud Completion

Shumeng Li, Jian Zhang, Lei Qi, Yinghuan Shi

Abstract: Segment Anything Model (SAM) adaptation has shown remarkable performance in medical image segmentation, but typically relies on large and precisely annotated datasets. However, acquiring such dense annotation is a labor-intensive and time-consuming task that requires significant expertise. An effective direction is to focus on sparse annotation, where only a few slices are annotated. However, sparse annotations are insufficient for capturing the complete 3D anatomical structure. To address this limitation, we innovatively leverage point cloud completion to generate robust volumetric shape from sparse annotation, offering a promising solution to this challenge. In this paper, we propose a novel Geometry-Aware SAM adaptation framework (namely GA-SAM) that integrates point cloud shape generation module with cross-view segmentation supervision mechanism. Specifically, we train a point cloud completion network to infer the 3D structure of the target anatomy. The generated point cloud shapes are then used to produce pseudo-labels, guiding the adaptation of SAM via a geometry-aware shape constraints. Furthermore, we incorporate a cross-view supervision mechanism, leveraging multi-view consistency to ensure reliable segmentation across different planes. We demonstrate the effectiveness of our method on Pancreas-CT dataset, surpassing the state-of-the-art SAM adaptation method by a Dice score of 15.25% and significantly improving segmentation robustness. Our code is available at https://github.com/ShumengLI/GA-SAM.

External IDs:dblp:conf/miccai/LiZQS25