Implementation of a closed-loop BCI system for real-time speech synthesis under clinical constraints

Kevin Meng, Euiyoung Kim, Simon J. Vogrin, Mark J. Cook, Farhad Goodarzy, David B. Grayden, Chun Kee Chung

Abstract: Closed-loop brain-computer interface (BCI) systems that provide real-time feedback to their users are essential for the synthesis of attempted or imagined speech from intracranial recordings. Here, we describe the implementation of our BCI speech synthesis system, which can be trained with a limited amount of overt speech to produce a continuous stream of audio outputs during subsequent speech imagery tasks. We evaluate (1) the effect of parameter choices on the execution time of individual operations in the BCI loop and (2) the accuracy of predicted outputs. To confirm the feasibility of our approach, we conduct simulations in a pseudo-prospective fashion using recorded datasets from five patients undergoing intracranial epilepsy monitoring. We propose that our system can be used to synthesize different types of speech under specific clinical constraints.