Graph-Enhanced EEG-to-Text Decoding: A Spatio-Temporal Relational Embedding Framework for Brain Signal Translation

Larine Ouyang

Keywords: EEG-to-Text Decoding, Brain–Computer Interfaces, Graph Neural Networks

TL;DR: We propose the first graph-enhanced EEG-to-text decoding framework that uses spatio-temporal relations among electrodes, which outperforms RNN and Transformer baselines.

Abstract: Despite recent progress in brain–computer interfaces (BCIs), decoding natural language directly from EEG remains a critical challenge. Existing EEG-to-text models primarily treat signals as sequential time series, which severely limits their ability to capture the spatial and temporal relationships among electrodes and limits the possibility of generalization in low-data regimes. To address this challenge, we propose a novel graph-enhanced framework to explicitly model relational information in brain signals. The key idea of our framework is to construct Spectro-Topographic Relational Graphs (STRG) that jointly encode static electrode topology and dynamic inter-channel functional connectivity. From these graphs, we derive Spatio-Temporal Relational Embeddings (STRE), which provide graph-aware representations for downstream sequence-to-sequence decoding. Specifically, (i) STRG captures spatial adjacency and frequency-specific connectivity, (ii) STRE transforms these relational structures into embeddings aligned with text decoding, and (iii) the overall framework integrates these embeddings with a neural decoder to generate natural language outputs. To the best of our knowledge, this is the first graph-enhanced approach for EEG-to-text decoding that explicitly uses graph-based representations of EEG signals. Empirical results show that our framework delivers substantial improvements over strong recurrent and Transformer baselines. In particular, our Graph-Enhanced EEG-to-Text Decoding achieves up to 16% relative gains on BLEU-4, which highlights the effectiveness of relational graph modeling for advancing neural decoding.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 23037