Handwriting decoding as a challenging Motor Imagery task for EEG Foundation Models

Srinivas Ravishankar, Virginia R. de Sa

Keywords: BCI, EEG, decoding, motor, handwriting

TL;DR: First work demonstrating imagined handwriting decoding from EEG. Presents a more challenging benchmark for Foundation Models that have demonstrated SOTA in simpler MI tasks.

Abstract:

Foundation Models (FMs) for EEG have achieved state-of-the-art performance on multiple Motor Imagery (MI) datasets, indicating their potential to provide robust, generalizable representations across diverse contexts. In this work, we investigate handwriting decoding as a challenging MI task to evaluate the generalizability of FMs for EEG. Recent studies in handwriting decoding have reported higher-than-chance performance in decoding handwritten letters from EEG. However, all prior works have attempted to decode handwriting from EEG during actual motion. Furthermore, they assume that precise movement-onset is known. We introduce a setting closer to real-world use where either movement-onset is not known or movement does not occur at all, fully utilizing motor imagery. Crucially, we find that current FMs for EEG, despite showing SOTA performance in multiple MI datasets (hand vs foot classification) are outperformed by a task-specific EEGNet model in this fine-grained task.
In parallel, we also investigate avenues that are most promising for improving decoding performance. In our 4-letter classification task, we show that (a) Knowledge of movement-onset is crucial to reported decoding performance in prior works, with average performance across subjects dropping from $41.4\%$ to $32.4\%$. (b) Increasing test-time signal quality provides significant performance improvements ($45\%$ to $78\%$ in our best subject) compared to scaling training data with single-trial EEG. (c) Fully imagined handwriting can be decoded from EEG with higher-than-chance performance.

Submission Number: 69