Neural encoding of multidimensional handwriting movements for brain-computer interfaces

Zebin Wang, Guangxiang Xu, Yueming Wang, Yaoyao Hao

Abstract: Neural representations of handwriting persist even years after paralysis, which was previously employed to build high performance brain-computer interfaces (BCI) for brain-to-text communication. However, handwriting was traditionally viewed as two-dimensional (2D) movements, ignoring other key kinematic and kinetic dimensions. How brain encodes these additional dimensions during handwriting and the potential of this information to enhance BCI performance were largely unexplored. Here, we recorded intracortical neural signals from a paralyzed individual engaged in handwriting imaginary and tried to align these signals with the multidimensional handwriting movements from healthy subjects. We achieved successful decoding of neural signals from the paralyzed individual, enabling reconstruction of 2D handwriting trajectories with high accuracy. We then characterized handwriting from heathy subjects as multidimensional movements which include 3D velocity of pen tip, pen grip strength, pen tip pressure on the paper, and 8-channel electromyography (EMG) on forearm. Employing a neural encoding model, we found that more variance of the neural signals could be accounted by these additional variables, suggesting brain encodes handwriting in multiple dimensions rather than solely in 2D. Together, these results shed light on the complex neural mechanisms underlying handwriting, offering new insights into the potential for improving related BCI applications.