Validation of the Role of Attention Mechanism in Predicting Brain Activity

Haruka Taguchi, Satoshi Nishida, Shinji Nishimoto, Ichiro Kobayashi

Abstract: In this study, we estimate the state of the human brain from visual stimuli by regressing the brain activity state from image features extracted from an image identification deep learning model. We introduce Attention Branch Network, which enhances to capture the features of the identified target by attention when extracting image features, into the image identification deep learning model and estimate the brain activity state from the image features weighted or unweighted by attention. Through experiments, we aim to verify the role of attention mechanism in estimating brain activity state from visual stimuli. As a result, we confirmed that the introduction of Attention did not have a significant effect on the estimation accuracy, but that differences were observed in the areas where the estimation accuracy was higher.