EpilepsyFM: Foundation Model for Learning Generalized Epileptic Representations from EEG and SEEG Signals
Keywords: Epilepsy, Foundation Model, Electroencephalography, Stereoelectroencephalography
Abstract: Extracranial electroencephalography (EEG) and intracranial stereoelectroencephalography (SEEG) are crucial for epilepsy diagnosis. However, existing deep learning models often limit themselves to specific signal types and application scenarios, leading to challenges in generalization and perception capabilities. While large language models excel in natural language processing, they cannot effectively capture the disease-specific signal features in the highly specialized field of epilepsy, and the lack of pre-training data restricts their generalization ability. To address these issues, we propose a Epilepsy Foundation Model (EpilepsyFM), a domain-specific foundational model that considers the mechanisms of seizure and propagation in epilepsy. EpilepsyFM learns a generalized representation of epilepsy through unsupervised pre-training across various signal types, data formats, and sources, and optimizes multiple epilepsy-related downstream tasks through fine-tuning. We collected clinical EEG and SEEG data from multiple patients at a first-class hospital, as well as the currently largest publicly available epilepsy dataset, the TUH series, ensuring diversity in representation learning. First, the neural activity signals are segmented into multiple patches, and a discrete EEG and SEEG neural tokenizer is trained to construct a domain-specific neural codebook for epilepsy. Then, EpilepsyFM takes into account the mechanisms of clustered neuronal discharges in epilepsy and designs a channel set masking strategy to enhance the model's ability to capture the spatiotemporal characteristics of the signals. The model fully utilizes the multi-dimensional propagation characteristics of seizures through temporal, spectral, and spatial encoder modules, achieving comprehensive representation of complex neural signals. Extensive experiments show that EpilepsyFM achieves state-of-the-art performance in a variety of domain-specific tasks, including seizure detection and both short-term and long-term predictions of neural signals, demonstrating strong generalization ability and broad clinical application potential.
Primary Area: foundation or frontier models, including LLMs
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 10217
Loading