Elementary, My Dear Watson: Non-Invasive Neural Keyword Spotting in the LibriBrain Dataset

Gereon Elvers, Gilad Landau, Oiwi Parker Jones

TL;DR: We introduce a reproducible Neural Keyword Spotting benchmark for non-invasive BCIs on LibriBrain with PR-centric metrics, open code/tutorials, log-linear scaling and word-level effects, making it a practical, privacy-preserving control channel.

Abstract: Non-invasive brain-computer interfaces (BCIs) are beginning to benefit from large, public benchmarks. However, most shared tasks currently target simpler, foundational tasks like speech detection, while useful results on more practical tasks like phoneme classification or full word detection remain elusive. We propose keyword spotting (KWS) as a practically applicable, privacy-aware intermediate task. Using the 52-hour single-participant LibriBrain corpus, we provide standardised train/validation/test splits for reproducible benchmarking, and adopt an evaluation protocol tailored to extreme class imbalance: area under the precision-recall curve (AUPRC) as a robust evaluation metric, complemented by false alarms per hour (FA/h) at fixed recall to capture user-facing trade-offs. We release a modified version of the \texttt{pnpl} library with word-level dataloaders and Colab-ready tutorials to simplify deployment and further experimentation, to be complemented with a public-facing leaderboard in the future. As an initial reference point we report a compact 1-D Conv/ResNet baseline with focal loss and top-$k$ pooling that is trainable on a single consumer-class GPU and achieves $\approx 13\times$ the permutation-baseline AUPRC on held-out sessions. Analyses reveal (i) predictable within-subject scaling—performance improves log-linearly with more training hours—and (ii) word-level factors (frequency and duration) that systematically modulate detectability. KWS thus offers an immediately actionable one-bit control channel in the short term and, in the longer term, a privacy-preserving gate for future continuous brain-to-text systems.

Length: long paper (up to 8 pages)

Domain: methods

Author List Check: The author list is correctly ordered and I understand that additions and removals will not be allowed after the abstract submission deadline.

Anonymization Check: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and URLs that point to identifying information.

Submission Number: 69