MindAligner: Explicit Brain Functional Alignment for Cross-Subject Visual Decoding from Limited fMRI Data

Yuqin Dai, Zhouheng Yao, Chunfeng Song, Qihao Zheng, Weijian Mai, Kunyu Peng, Shuai Lu, Wanli Ouyang, Jian Yang, Jiamin Wu

TL;DR: We propose MindAligner, an explicit functional alignment framework for cross-subject brain decoding with limited fMRI data.

Abstract: Brain decoding aims to reconstruct visual perception of human subject from fMRI signals, which is crucial for understanding brain's perception mechanisms. Existing methods are confined to the single-subject paradigm due to substantial brain variability, which leads to weak generalization across individuals and incurs high training costs, exacerbated by limited availability of fMRI data. To address these challenges, we propose MindAligner, an explicit functional alignment framework for cross-subject brain decoding from limited fMRI data. The proposed MindAligner enjoys several merits. First, we learn a Brain Transfer Matrix (BTM) that projects the brain signals of an arbitrary new subject to one of the known subjects, enabling seamless use of pre-trained decoding models. Second, to facilitate reliable BTM learning, a Brain Functional Alignment module is proposed to perform soft cross-subject brain alignment under different visual stimuli with a multi-level brain alignment loss, uncovering fine-grained functional correspondences with high interpretability. Experiments indicate that MindAligner not only outperforms existing methods in visual decoding under data-limited conditions, but also provides valuable neuroscience insights in cross-subject functional analysis. The code will be made publicly available.

Lay Summary:

fMRI decoding is like mind-reading: it reconstructs what someone sees based on their brain scans. However, because each person’s brain response patterns vary so much, and because we usually have only a handful of scans, training a single model that works well for everyone is both difficult and expensive.

To overcome this, we developed MindAligner, a method that
1. learns to translate a new subject’s brain signals into a shared, “universal” format,
2. aligns those translated signals to patterns drawn from other people viewing different images,
3. plugs the aligned signals into an existing decoder to predict what the person actually saw.

Even when given very little data from a new participant, MindAligner maintains high decoding accuracy. By making it practical to reuse pre-trained models across individuals, our approach promises to accelerate both basic neuroscience research and real-world medical applications.

Application-Driven Machine Learning: This submission is on Application-Driven Machine Learning.

Link To Code: https://github.com/Da1yuqin/MindAligner

Primary Area: Applications->Neuroscience, Cognitive Science

Keywords: Brain Decoding, Functional Alignment, Cross-subject Decoding, Neuroscience, Neuroimaging, Visual Perception

Submission Number: 3639