Zero-Shot Performance of the Segment Anything Model (SAM) in 2D Medical Imaging: A Comprehensive Evaluation and Practical Guidelines

Christian Mattjie de Oliveira, Luis Vinícius de Moura, Rafaela Cappelari Ravazio, Lucas S. Kupssinskü, Otávio Parraga, Marcelo Mussi Delucis, Rodrigo C. Barros

Abstract: This study evaluates the potential of the “Segment Anything Model” (SAM) as a robust alternative for medical imaging segmentation in a zero-shot learning context. We evaluate SAM's performance across six diverse medical imaging datasets spanning four different imaging modalities. By employing eight unique prompting strategies we reveal comprehensive insights into SAM's adaptability. The Bounding Box strategy, with its variations, matched or even outperformed existing benchmarks. On the Breast Ultrasound Images dataset, SAM notably outperformed SOTA models, attesting to its capability as a robust zero-shot segmentation tool. Conversely, challenges arose with datasets having indistinct boundaries and inconsistent annotations, as in skin lesion images. The study also establishes a practical set of guidelines aimed at optimizing SAM's clinical usage. The findings underscore SAM's potential as a powerful, versatile tool for medical imaging segmentation, alleviating the burden of manual segmentation and potentially improving ground truth masks for labeling new datasets. With its minimal resource requirements and promising results, SAM represents an exciting advancement in medical imaging analysis.