Zero-shot capability of 2D SAM-family models for bone segmentation in CT scans
Keywords: Segment anything model, Medical image segmentation, Foundation models, Bone segmentation
TL;DR: We use non-iterative prompting strategies composed of bounding box, points and combinations to test the zero-shot capability of 2D SAM-family models for bone CT segmentation on four different skeletal regions.
Abstract: The Segment Anything Model (SAM) and similar models build a family of promptable foundation models (FMs) for image and video segmentation. The object of interest is identified using prompts---user provided input such as bounding boxes or points---and the models have shown very promising results when it comes to generalization to new tasks. However, extensive evaluation studies are required for medical applications, to assess their strengths and weaknesses in clinical settings. As the performance of those models is highly dependent on the quality and quantity of their prompts, it is necessary to thoroughly benchmark the different options. Currently, no dedicated evaluation studies exist specifically for bone segmentation in CT scans. Leveraging high-quality private and public datasets on four skeletal regions, we test the zero-shot capabilities of SAM-family models for bone CT segmentation, using non-interactive prompting strategies, composed of bounding box, points and combinations of the two. Additionally, we design a guideline for informed decision-making in 2D non-interactive prompting based on our insights on segmentation performance and inference time. Our results show that SAM and SAM2 currently outperform medically fine-tuned FMs, and prompted with a bounding box together with a center point have the best performance across all tested settings.
Primary Subject Area: Foundation Models
Secondary Subject Area: Segmentation
Paper Type: Validation or Application
Registration Requirement: Yes
Submission Number: 91
Loading