What Fine-Tuning Changes: A Radiomic Lens on Prostate Foundation Model Representations

Yipei Wang, Yaxi Chen, Wen Yan, Natasha Thorley, Alexander Ng, Dean C. Barratt, Daniel C. Alexander, Shonit Punwani, Mark Emberton, Veeru Kasivisvanathan, Yipeng Hu

Keywords: mpMRI; Foundation model; Radiomic feature; Interpretability

Abstract: Clarifying how foundation model encoders change during fine-tuning is important for transparency and trustworthiness in their medical imaging applications. It may also be useful for further understanding, developing and adapting these models. However, the latent representations produced by such encoders are high dimensional and lack explicit semantic meaning, making it difficult to characterise how task-specific adaptation modifies them. In this study, we introduce a radiomics-based framework that provides an interpretable lens through which these representational changes can be examined and often better understood. Using prostate cancer patient imaging data, we train a two-layer MLP to learn the relationship between radiomic descriptors and encoder embeddings prior to fine-tuning. This model captures non-linear associations through its first layer, while the final linear layer offers an interpretable mapping from radiomic attributes to (transformed) latent features. To quantify the effect of fine-tuning, the first layer is fixed, and only the linear layer is re-estimated using the embeddings from the fine-tuned encoder. Comparing the pre- and post-fine-tuning linear weights yields a direct quantitative measure of how the encoder’s emphasis on specific radiomic characteristics shifts during fine-tuning. We validate the approach using a prostate MRI foundation model and multiple downstream tasks. The analysis reveals consistent, task-dependent changes in the encoder’s sensitivity to radiomic texture and intensity features. This work provides the first radiomics-based methodology for systematically interpreting how fine-tuning restructures foundation model representation in medical imaging. The implementation is available at: https://github.com/pipiwang/RaiomicLens

Primary Subject Area: Interpretability and Explainable AI

Secondary Subject Area: Foundation Models

Registration Requirement: Yes

Reproducibility: https://github.com/pipiwang/RaiomicLens

Read CFP & Author Instructions: Yes

Originality Policy: Yes

Single-blind & Not Under Review Elsewhere: Yes

LLM Policy: Yes

Submission Number: 181