Adapting Segment Anything Models to Medical Imaging via Fine-Tuning without Domain Pretraining
Track: Traditional track
Keywords: Segment Anything, Medical Pre-training, Domain Adaptation
Abstract: Medical image segmentation is an important task in the context of medical care, with applications in diagnostic and treatment processes. Segment Anything (SAM), a generalist foundation model trained on a corpus of 11 million natural images, demonstrates limited adaptability to the medical domain in a zero-shot prompting context, but shows promise under parameter-efficient fine-tuning. MedSAM is a foundation model which adapts SAM to the medical domain via training on a diverse medical corpus consisting of different modalities (one million images of modality CT, MRI, CXR, etc). In this work, we evaluate the advantage of MedSAM over SAM for medical task-specific adaptation achieved via parameter-efficient fine-tuning. Our results demonstrate that MedSAM does not yield a consistent advantage over SAM in this setting. We also introduce a novel parameter-efficient approach, LoRaMedNet, which combines elements of previous fine-tuning methods to achieve greater flexibility of adaptation for SAM, and find that LoRaMedNet-adapted SAM attains the best performance. The implication of this finding is that generalist models like SAM can achieve superior adaptation to specific medical tasks even when compared to models with medical pre-training.
Presentation And Attendance Policy: I have read and agree with the symposium's policy on behalf of myself and my co-authors.
Ethics Board Approval: No, our research does not involve datasets that need IRB approval or its equivalent.
Data And Code Availability: No, we will not be making any data and/or code public.
Primary Area: Clinical foundation models
Student First Author: Yes, the primary author of the manuscript is a student.
Submission Number: 22
Loading