Dynamic Prompt Generation for Interactive 3D Medical Image Segmentation

Tidiane Camaret Ndir, Alexander Pfefferle, Robin Tibor Schirrmeister

Keywords: Interactive segmentation, 3D medical imaging, Dynamic prompt generation

TL;DR: We propose a training strategy based on dynamic prompt generation for deep-learning based interactive 3d segmentation models

Abstract: Interactive 3D biomedical image segmentation requires efficient models that can iteratively refine predictions based on user prompts. Current foundation models either lack volumetric awareness or suffer from limited interactive capabilities. We propose a training strategy that combines dynamic volumetric prompt generation with content-aware adaptive cropping to optimize the use of the image encoder. Our method simulates realistic user interaction patterns during training while addressing the computational challenges of learning from sequential refinement feedback on a single GPU. For efficient training, we initialize our network using the publicly available weights from the nnInteractive segmentation model. Evaluation on the \textbf{Foundation Models for Interactive 3D Biomedical Image Segmentation} competition demonstrates strong performance with an average final Dice score of 0.6385, normalized surface distance of 0.6614, and area-under-the-curve metrics of 2.4799 (Dice) and 2.5671 (NSD).

Submission Number: 12