MTSSRL-MD: Multi-Task Self-Supervised Representation Learning for EEG Signals across Multiple Datasets

ICLR 2026 Conference Submission24689 Authors

Keywords: Channel alignment, Cross-dataset generalization, Electroencephalography, Heterogeneous EEG datasets, Multi-dataset pre-training, Multi-task learning, Sleep stage classification, Self-supervised learning, Uncertainty-weighted loss

TL;DR: MTSSRL-MD integrates multi-dataset and multi-task self-supervised representation learning with channel alignment to enable efficient, interpretable, and generalizable EEG sleep staging under label scarcity.

Abstract: Electroencephalography (EEG) supports diverse clinical applications. However, effective EEG representation learning remains difficult because scarce label annotations and heterogeneous EEG montages limit the scale of available datasets. In practice, single and small-scale datasets often result in models with poor generalization, particularly for underrepresented classes with limited samples, which are harder to learn reliably. These challenges become even more critical in the EEG-based sleep stage classification task, especially for minority stages that are not only scarce but also transitional with overlapping characteristics, which makes them prone to misclassification. In this work, we propose MTSSRL-MD (Multi-Task Self-Supervised Representation Learning for EEG Signals across Multiple Datasets), a unified framework that combines multi-dataset and multi-task self-supervised pretraining with a channel alignment module to alleviate the impact of scarce labels, heterogeneous EEG montages, and small-scale datasets that often cause poor generalization. This design enables the learning of EEG representations that are generalizable. Multi-dataset learning provides broader feature diversity that facilitates more robust cross-dataset generalization. A spatial-attention Channel Alignment Module (CAM) projects heterogeneous EEG montages into a shared channel space and provides spatial weights that highlight regions aligned with standard EEG montages, offering interpretability. Complementary self-supervised tasks—augmentation contrastive, temporal shuffling discrimination, and frequency band masking—provide temporal and spectral information that improve robustness on these underrepresented classes. Experiments on three heterogeneous EEG sleep datasets show that MTSSRL-MD consistently outperforms single-dataset SSRL baselines and even surpasses SeqCLR, a representative multi-dataset single-task SSRL method, particularly under low-label conditions, demonstrating the effectiveness of integrating multi-dataset and multi-task learning for EEG-based sleep stage classification. Besides classification performance, MTSSRL-MD achieves more efficient inference than single- and multi-dataset SSRL baselines. Moreover, the unified design of our proposed method allows the use of a single pretrained encoder to be fine-tuned across diverse datasets, highlighting efficiency and practical value for clinical research, suggesting strong potential for deployment in real-world settings.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 24689