Steady Progress Beats Stagnation: Mutual Aid of Foundation and Conventional Models in Mixed Domain Semi-Supervised Medical Image Segmentation

Qinghe Ma, Jian Zhang, Zekun Li, Lei Qi, Qian Yu, Yinghuan Shi

Abstract:

Large pretrained visual foundation models exhibit impressive general capabilities. However, the extensive prior
knowledge inherent in these models can sometimes be a
double-edged sword when adapting them to downstream
tasks in specific domains. In the context of semi-supervised
medical image segmentation with domain shift, foundation
models like MedSAM tend to make overconfident predictions, some of which are incorrect. The error accumulation hinders the effective utilization of unlabeled data
and limits further improvements. In this paper, we introduce a Synergistic training framework for Foundation and
Conventional models (SynFoC) to address the issue. We observe that a conventional model trained from scratch has the
ability to correct the high-confidence mispredictions of the
foundation model, while the foundation model can supervise it with high-quality pseudo-labels in the early training
stages. Furthermore, to enhance the collaborative training
effectiveness of both models and promote reliable convergence towards optimization, the consensus-divergence consistency regularization is proposed. We demonstrate the
superiority of our method across four public multi-domain
datasets. In particular, our method improves the Dice score
by 10.31% on the Prostate dataset. Our code is available at
https://github.com/MQinghe/SynFoC.