Advancing SAM for Dental Imaging: A Detection-Prompted Pipeline for High-Accuracy Tooth Segmentation

Saya Atchibay, Aiya Alchaar, Peixi Liao, Farshid Alizadeh-Shabdiz

Keywords: Dental Image Segmentation, Object Detection, X-Ray images, Transfer learning, Foundation Model, SAM, YOLOv11, U-Net

TL;DR: A YOLO-guided, U-Net–refined SAM pipeline achieves high-accuracy tooth segmentation on panoramic X-rays without retraining SAM

Abstract: Since the Segment Anything Model (SAM) was introduced in 2023 (Kirillov et al., 2023) numerous studies have investigated its performance on medical imaging datasets. SAM has been trained on the SA-1B dataset, which comprises 11M natural images and 1.1B segmentation masks, which makes it a very powerful engine. However, due to the differences between natural images and medical X-ray images, SAM does not perform as well as much smaller CNN based models for segmentation of medical images. To improve segmentation of dental panoramic radiographs, we propose a detection-guided prompting pipeline in which YOLOv11 localizes tooth regions and SAM generates initial masks. Although these SAM masks capture the general tooth structure, they often miss fine morphological details. To address this, we introduce a lightweight U-Net refinement module that operates purely as a post-processing step, correcting local boundary errors without fine-tuning SAM. Trained on our combined dataset, YOLOv11 achieved an mAP@0.5 of 0.9931. Our full pipeline improves SAM’s zero-shot Dice score from 0.672 to 0.9026, demonstrating that detection-guided prompting coupled with lightweight refinement substantially enhances segmentation quality.

Primary Subject Area: Transfer Learning and Domain Adaptation

Secondary Subject Area: Learning with Noisy Labels and Limited Data

Registration Requirement: Yes

Visa & Travel: Yes

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 334