Diffusion-driven Dual-flow Source-Free Domain Adaptation for Medical Image Segmentation

Wei Zhou, Jianhang Ji, Wei Cui, Yugen Yi, Yixin Chen

Abstract: Source-Free Domain Adaptation (SFDA) aims to adapt a pre-trained model to unlabeled target domain data without access to source domain data, presenting a significant challenge for medical image segmentation. Most current approaches address this challenge through self-training, employing manually augmented target domain images and pseudo-labels to enforce consistency regularization. However, these approaches still encounter two primary issues. Firstly, manually augmented consistency self-training results in performance degradation due to the semantic mismatch between the target domain images and noisy pseudo-labels. Secondly, they fail to fully exploit the informative content present in the target domain, exhibiting inadequate adaptability, particularly in significant domain gaps. To address these, we introduce the Diffusion-driven Dual-flow SFDA (D2SFDA), the pioneering framework to integrate a diffusion model into SFDA for medical image segmentation. Our D2SFDA framework comprises two novel components: the Diffusion Perturbation Flow (DPF) and the Twin-Knowledge Investigation Flow (TKIF). DPF utilizes pseudo-labels to generate diverse and semantically consistent diffusion views, providing more realistic supervision, potentially enhancing model stability. Surprisingly, DPF using only diffusion images outperforms self-training using real images, as evidenced by the superior average Dice score on the BASE1 target domain of the RIGA+ dataset (90.31% vs. 85.64%). Additionally, TKIF rigorously analyzes the target domain with dual-focus consistency regularization on domain-invariant and target domain-specific knowledge, effectively reducing domain gaps, resulting in an improvement from 90.31% to 91.79%. Extensive experiments on two cross-domain datasets confirm that our D2SFDA surpasses state-of-the-art SFDA approaches in effectively addressing domain shift issues. The code is available at https://github.com/M4cheal/D2SFDA.