TTA-FM: Patient-Specific Test-Time Adaptation Using Foundation Models for Improved Prostate Segmentation in Magnetic Resonance Images

Hariharan Ravishankar, Prasad Sudhakar, Phaneendra K. Yalavarthy

Abstract: This study proposes a fully automated framework TTA-FM: for patient-specific, test-time adaptation of pre-trained prostate segmentation models using vision foundation models in magnetic resonance (MR) images. The proposed frame-work utilizes salient properties of foundation models and task-specific models to filter, impute, and generate high-confidence pseudolabels for model adaptation. Existing test-time adaptation methods are often not patient-specific, require strong anatomical priors or modify training procedure rendering them impractical. Vision foundation models often fail to perform adequately on medical images due to differences in texture and shape properties of objects of interest. The proposed TTA-FM framework designed to overcome these limitations, outperforms both sets of methods for prostate segmentation from MR images. The TTA-FM method was evaluated on a challenging cross-site domain adaptation setting and achieves state-of-the-art results with no additional constraints, making it first of its kind adaptation framework. Further, results from a set of ablation studies are reported, demonstrating value of each step of the proposed method.