Enhanced EEG-fNIRS Classification through Concatenated Convolutional Neural Network with Band Analysis

Chayut Bunterngchit, Jiaxing Wang, Thanaphon Chearanai, Zeng-Guang Hou

Abstract: Brain-computer interfaces (BCIs) have emerged as a groundbreaking technology enabling direct communication between the human brain and external devices. Yet, the attainment of higher accuracy rates while making classifications remains a problem and demands the development of more sophisticated machine learning architectures and the exploration of novel feature domains. To address these issues, we have presented a novel approach for enhanced EEG-fNIRS (electroencephalography-functional near-infrared spectroscopy) classification through a concatenated convolutional neural network (CNN) with band analysis. Our proposed methodology leverages the power spectral density (PSD) of EEG signals in specific frequency bands and captures metabolic changes in the brain through fNIRS data. Then it employs a concatenated CNN architecture for improved feature extraction and classification in EEG-fNIRS data. Through an experimental evaluation on a public dataset containing three mental workload (MWL) tasks, our proposed model demonstrates superiority compared to previous studies by achieving higher accuracy rates of 93.8%, 97.2%, and 94.5% for n-back, discrimination/selection response (DSR), and word generation (WG) tasks, respectively. Furthermore, our results emphasize the advantages of utilizing a hybrid EEG-fNIRS approach over unimodal EEG or fNIRS methods. These outcomes underscore the potential of our multimodal EEG-fNIRS approach and its applicability in BCI, neurorehabilitation, and human-computer interaction contexts.