Personalized Visual Scanpath Prediction Using IOR-ROI Weighted Attention Network

Feng Yuan, Matthieu Perreira Da Silva, Alexandre Bruckert

Abstract: Predicting visual scanpaths plays an important role in modeling overt human visual attention and search behavior. Due to the rapid development of deep learning, previous scanpath prediction models have made significant progress. However, these methods only focus on common visual saliency, while ignoring differences in observational traits between individuals. Therefore, we propose a more challenging task, which is to provide personalized scanpath prediction for different subjects. In response to the above tasks, we designed a personalized scanpath prediction model with two branches. Specifically, we use the visual hit branch to realize the interaction between multiple features, obtain high-dimensional features with rich subject information, and standardize the appearance location of fixations at the same time. Then, weighted attention in scanpath prediction branch fuses image embedding and subject embedding to obtain an ROI map more similar to the observed characteristics of the subject using high-dimensional features obtained from the visual hit branch.