From Prediction to Prompt: Leveraging nnU-Net Outputs to Guide SAM for Active Learning in 3D Dental Segmentation

Nicolas Martin, Jean-Pierre Chevallet, Philippe Mulhem

Keywords: Active Learning, Unet, Segment Anything, CBCT, Segmentation

TL;DR: nnU-Net Predictions as Prompts for SAM-Med3D

Abstract:

To enhance annotation efficiency in 3D dental Cone Beam Computed Tomography (CBCT) image segmentation, this paper explores an active learning approach that leverages nnU-Net predictions to generate prompts for a specialized 3D Segment Anything Model (SAM).
The objective is to minimize the annotation burden without relying on prompts during the inference phase.
First, our experiments showed that AL offers similar segmentation performance with less than 20% of the original annotations.
Second, random selection offers similar results than more complex sampling method with less more computing demand.
Third, the predictions of nnU-Net on unannotated images provided effective prompts for the SAM model specialized in 3D medical images. Combining these two approaches reduced the required amount of manual annotation by up to 50%.
This paper paves the way for more easily obtaining new annotated datasets in the dental domain while simultaneously training a segmentation model, by leveraging SAM-like models.

Changes Summary:

We thank the reviewers for their valuable comments and suggestions. Based on their feedback, we have revised the paper accordingly. The main changes are as follows:
- Corrected minor typos, unclear sentences, and small errors in the bibliography.
- Revised captions of figures and tables to make them self-explanatory.
- Updated figures to improve consistency and clarity.
- Added missing parameter details to enhance replicability.
- Improved the description of the applied workflows.
- Add qualitative visualization of AL rounds
- Added new references.

Main Tex File: paper.tex

Confirm Latex Only: true

Code Url: https://github.com/martinicmrim/sam_nnunet

Authors Changed: false

Submission Number: 5