Efficient Multi Subject Visual Reconstruction from fMRI Using Aligned Representations

ICLR 2026 Conference Submission17225 Authors

Keywords: Computational Neuroscience, fMRI, Neuroimaging, Diffusion, CLIP, alignment, neuroAI

TL;DR: We introduce a training paradigm for brain signal to image reconstruction by developing an aligned and shared cross-subject representation space that is especially powerful in low-data settings

Abstract: Reconstructing visual images from fMRI data is a central but highly challenging problem in neuroscience. Despite recent progress, current methods fall short when data and computation are limited—--precisely the conditions under which this task is most critical. We introduce a novel architecture-agnostic training paradigm to improve fMRI-based visual reconstruction through a subject-agnostic common representation space. We show that it is possible to leverage subject-specific lightweight modules to develop a representation space where different subjects not only lie in a shared space but are also aligned semantically. Our results demonstrate that such a training pipeline achieves significant performance gains in low-data scenarios. We supplement this method with a novel algorithm to select the most representative subset of images for a new subject. Using both techniques together, one can fine-tune with at most 40\% of the data while outperforming the baseline trained with the minimum standard dataset size. Our method generalizes across different training paradigms and architectures, producing state-of-the-art performance and demonstrating that a subject-agnostic aligned representation space is the next step towards efficient Brain-Computer Interfaces.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 17225