Understanding the brain with attention: a survey of transformers in brain sciences

Cheng Chen, Huilin Wang, Yunqing Chen, Xinye Yang

Abstract:

Owing to their superior capabilities and advanced achievements, Transformers have gradually attracted attention with regard to understanding complex brain processing mechanisms. Thisstudy aimsto comprehensively review and discussthe applications of Transformersin brain sciences. First, we present a brief introduction of the critical architecture of Transformers. Then, we overview and analyze their most relevant applicationsin brain sciences, including brain disease diagnosis, brain age prediction, brain anomaly detection, semantic segmentation, multi-modal registration, functional MagneticResonance Imaging (fMRI)modeling, Electroencephalogram(EEG) processing, and multi-task collaboration. We organize the model details and open sources for reference and replication. In addition, we discuss the quantitative assessments, model complexity, and optimization of Transformers, which are topics of great concern in the field. Finally, we explore possible future challenges and opportunities, exploiting some concrete and recent cases to provoke discussion and
innovation. We hope that this review will stimulate interest in further research on Transformers in the context of brain sciences.