EEG-DINO: Learning EEG Foundation Models via Hierarchical Self-distillation

Xujia Wang, Xuhui Liu, Xi Liu, Qian Si, Zhaoliang Xu, Yang Li, Xiantong Zhen

Abstract: Electroencephalography (EEG) provides a non-invasive window into the brain’s electrical activity, playing an essential role in various brain-computer interface (BCI) and healthcare applications. In this paper, we propose EEG-DINO, a novel foundation model for EEG encoding based on a hierarchical self-distillation framework. By multi-view semantic alignment, the model is able to extract multi-level semantic features from EEG data, which captures a wide range of semantic information, increasing the robustness against noise and variances inherent in complex EEG signals. Moreover, acknowledging the unique heterogeneous spatial-temporal dependencies in EEG signals, we design a channel-aware sampling mechanism and a decoupled positional embedding scheme. They independently address spatial and temporal dimensions, enabling the model to capture the intricate structural characteristics of EEG signals. We train EEG-DINO on a large-scale EEG corpus spanning over 9000 h, which consistently achieves state-of-the-art performance on multiple downstream tasks (The pre-trained weights and code for fine-tuning are available at: https://huggingface.co/eegdino/EEG-DINO). These results demonstrate the great effectiveness of our self-distillation framework for EEG encoding.

External IDs:dblp:conf/miccai/WangLLSXLZ25