BAR: Probing Brain Encoders with Concept-Based Explanations

Huadi Wang, Weihao Xia, Cengiz Oztireli

Keywords: Brain Encoder, Concept-Based Explanation, Feature Attribution, Brain-to-Image Reconstruction, Brain Function

TL;DR: We propose the Brain Activation Region (BAR) framework to investigate human-interpretable concepts learned by brain encoders and input features contributing to this learning.

Abstract: Brain encoders have demonstrated promising capabilities in extracting semantic features from brain activity. However, the internal computations of these models remain largely opaque, which limits their adoption in critical brain research and applications. To address this challenge, we propose the Brain Activation Region (BAR) framework to investigate human-interpretable concepts learned by brain encoders and input features contributing to this learning. Specifically, we train kernel-based probes in the latent spaces of MindEye and UMBRAE, two state-of-the-art models that interpret viewed images from fMRI signals. We further apply a feature attribution approach to concept density functions, evaluating specific brain voxels and regions sensitive to visual semantics. Our trained classifiers demonstrate high accuracy across diverse visual and semantic concepts, effectively explaining the predictions made by brain encoders. Additionally, the feature attribution reveals two regions of interest (ROIs) associated with visual concept processing in human brains, aligning with findings in recent neuroscience research.

Submission Number: 14