UniMind: Unleashing the Power of LLMs for Unified Multi-Task Brain Decoding

Weiheng Lu, Chunfeng Song, Jiamin Wu, Pengyu Zhu, Yuchen Zhou, Weijian Mai, Chi Zhang, Qihao Zheng, Wanli Ouyang

Keywords: Brain-computer Interfaces, Brain Foundation Model, Large Language Model, Multi-task Learning

Abstract: Decoding human brain activity from electroencephalography (EEG) signals is a central challenge at the intersection of neuroscience and artificial intelligence, enabling diverse applications in mental state assessment, clinical monitoring, and human–machine interaction. Recent efforts have extensively investigated building EEG-based pretrained encoders for generalized brain decoding through large-scale training on multiple datasets. However, most of these approaches still struggle to achieve satisfactory performance without task-specific tuning, owing to the pronounced inherent heterogeneity across decoding tasks. To address these challenges, we present UniMind, a general-purpose EEG foundation model for unified multi-task brain decoding by uniquely unleashing the power of LLMs to comprehend complex neural patterns. UniMind enjoys several merits. First, we design a Neuro-Language Connector to transform the spatiotemporal neural patterns of EEG data into LLM-understandable representations. Second, a Task-aware Query Selection module is proposed to inject task-awareness into the cross-modal understanding by dynamically generating task-adaptive query tokens, enabling the learning of task-relevant neural patterns across diverse tasks. Extensive experiments across 10 datasets demonstrate that UniMind substantially outperforms state-of-the-art multi-task decoding models (11% gain on average), while also offering valuable neuroscientific insights into neural functional correlations across tasks. The code will be made publicly available.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 5659