BRENA: Brain-inspired Hierarchical Neural Alignment Framework for Visual Decoding from EEG Signals

Yanan Zhu, Ziwei Xiang, Jiamin Wu, Jinyang Guo, Hongyuan Zhang, Chunfeng Song, Hongjian Fang, Qihao Zheng, Yufei Guo, Xuelong Li, Xianglong Liu

Keywords: EEG decoding, BCI, Visual neural decoding, Object recognition

Abstract: Decoding human visual experiences from neural signals is crucial for understanding the relationship between brain activity and perceptual representations, driving the advancement of brain-computer interface (BCI) applications. Existing visual decoding methods typically adopt a global alignment paradigm for brain-visual alignment, which may not account for the human visual cortex’s region-specific selectivity where distinct cortical areas are selectively sensitive to different visual information. In this work, we propose BRENA, a BRain-inspired hiErarchical Neural Alignment framework by simultaneously aligning both region-level and global brain representations with visual embeddings for robust and accurate brain decoding. Unlike prior approaches that rely purely on global pooled representations, BRENA proposes an adaptive local neural alignment module to explore fine-grained correspondence between brain channel features and visual semantic units, allowing for better exploitation of brain signals by modeling region-specific feature selectivity. Additionally, a set of perceptual weights are adaptively generated to guide more target-aware alignment. We further integrate a global neural alignment module, rendering hierarchical brain-visual alignment with complementary region-level and global neural patterns captured. Experiments demonstrate that BRENA not only outperforms existing methods across subjects and settings but also reveals region-level brain selectivity for visual stimuli through meaningful local mappings between brain channels and diverse visual patterns.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 2019