Learning 3D Medical Image Models From Brain Functional Connectivity Network Supervision For Mental Disorder Diagnosis

Xingcan Hu, Wei Wang, Li Xiao

Keywords: 3D medical image, functional connectivity network, contrastive learning, mental disease diagnosis

Abstract:

For mental disorder diagnosis, most previous works are task-specific and focus primarily on functional connectivity network (FCN) derived from functional MRI (fMRI) data. However, the high cost of fMRI acquisition limits its practicality in real-world clinical settings. Meanwhile, the more easily obtainable 3D T1-weighted (T1w) MRI, which captures brain anatomy, is ofen overlooked in standard diagnostic processes of mental disorders.
To address these two issues, we propose CINP (Contrastive Image-Network Pre-training), a framework that employs contrastive learning between 3D T1w MRI and FCNs. CINP aims to learn a joint latent semantic space that integrates complementary information from both functional and structural perspective. During pre-training, we incorporate masked image modeling loss and network-image matching loss to enhance visual representation learning and modality alignment.
Furthermore, thanks to contrastive pre-training which facilitates knowledge transfer from FCN to T1w MRI, we introduce network prompting. This protocol leverages 3D T1w MRI from suspected patients and FCNs from confirmed patients for differential diagnosis of mental disorders.
Extensive experiments across three mental disorder diagnosis tasks demonstrate the competitive performance of CINP, using both linear probing and network prompting, compared with FCN-based methods and self-supervised pre-training methods.
These results highlight the potential of CINP to enhance diagnostic processes with the aid of 3D T1w MRI in real-world clinical scenario.

Primary Area: applications to neuroscience & cognitive science

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 10753