Functional Annotation of Human Cognitive States using Graph Convolution Networks

Yu Zhang, Pierre Bellec

TL;DR: Using a 10s window of fMRI signals, our GCN model identified 21 different task conditions from HCP dataset with a test accuracy of 89%.

Keywords: fMRI, functional connectivity, brain decoding, graph convolutional network, deep learning

Abstract: A key goal in neuroscience is to understand brain mechanisms of cognitive functions. An emerging approach is to study “brain states” dynamics using functional magnetic resonance imaging (fMRI). So far in the literature, brain states have typically been studied using 30 seconds of fMRI data or more, and it is unclear to which extent brain states can be reliably identified from very short time series. In this project, we applied graph convolutional networks (GCN) to decode brain activity over short time windows in a task fMRI dataset, i.e. associate a given window of fMRI time series with the task used. Starting with a populational brain graph with nodes defined by a parcellation of cerebral cortex and the adjacent matrix extracted from functional connectome, GCN takes a short series of fMRI volumes as input, generates high-level domain-specific graph representations, and then predicts the corresponding cognitive state. We investigated the performance of this GCN "cognitive state annotation" in the Human Connectome Project (HCP) database, which features 21 different experimental conditions spanning seven major cognitive domains, and high temporal resolution task fMRI data. Using a 10-second window, the 21 cognitive states were identified with an excellent average test accuracy of 89% (chance level 4.8%). As the HCP task battery was designed to selectively activate a wide range of specialized functional networks, we anticipate the GCN annotation to be applicable as a base model for other transfer learning applications, for instance, adapting to new task domains.