Mental Face Image Retrieval Based on a Closed-Loop Brain-Computer Interface

Nona Rajabi, Charles Chernik, Alfredo Reichlin, Farzaneh Taleb, Miguel Vasco, Ali Ghadirzadeh, Mårten Björkman, Danica Kragic

Abstract: Retrieval of mental images from measured brain activity may facilitate communication, especially when verbal or muscular communication is impossible or inefficient. The existing work focuses mostly on retrieving the observed visual stimulus while our interest is on retrieving the imagined mental image. We present a closed-loop BCI framework to retrieve mental images of human faces. We utilize EEG signals as binary feedback to determine the relevance of an image to the target mental image. We employ the feedback to traverse the latent space of a generative model to propose new images closer to the actual target image. We evaluate the proposed framework on 13 volunteers. Unlike previous studies, we do not restrict the possible attributes of the resulting images to predefined semantic classes. Subjective and objective tests validate the ability of our model to retrieve face images similar to the actual target mental images.