Better Representations, Better Speech BCIs: a Multitask Approach

Tommaso Boccato, Michał Olak, Jamie Oliver, Chadwick Boulay, Spencer Kellis, Matteo Ferrante

Track: Extended Abstract Track

Keywords: speech BCIs, MEA, neuroAI, representation learning

TL;DR: Representation-focused multitask training boosts speech BCI accuracy and enables the first near real-time neural-to-sentence decoder.

Abstract: Speech brain-computer interfaces represent a critical frontier for restoring communication in individuals with locked-in syndromes. While current approaches focus on scaling datasets and post-hoc language modeling, these strategies face immediate constraints due to data scarcity in neural recording settings. We propose a complementary path: enhancing representation learning through multitask training aligned with cortical encoding structure. Through systematic analysis of the largest intracortical speech dataset, we identify that acoustic features are most reliably encoded in speech motor cortex. Leveraging these insights, we develop multitask strategies combining phoneme prediction with acoustic regression and semantic objectives, achieving state-of-the-art performance: 13.7% word error rate. We also introduce the first end-to-end neural-to-sentence decoder for quasi-online communication. These results demonstrate that representation-first strategies substantially improve neural decoding within current data constraints while remaining compatible with future scaling efforts.

Submission Number: 35