Simulating Human Gaze with Neural Visual Attention

Leo Schwinn, Doina Precup, Bjoern Eskofier, Dario Zanca

Keywords: human attention, gaze modeling, eye-tracking, neural networks, biologically plausible, scanpath, human-like

TL;DR: We impose the constraint of foveated vision to pretrained neural networks and generate visual scanpaths; results are SOTA in unsupervised human visual attention modeling.

Abstract: Existing models of human visual attention are generally unable to incorporate direct task guidance and therefore cannot model an intent or goal when exploring a scene. To integrate guidance of any downstream visual task into attention modeling, we propose the Neural Visual Attention (NeVA) algorithm. To this end, we impose to neural networks the biological constraint of foveated vision and train an attention mechanism to generate visual explorations that maximize the performance with respect to the downstream task. We observe that biologically constrained neural networks generate human-like scanpaths without being trained for this objective. Extensive experiments on three common benchmark datasets show that our method outperforms state-of-the-art unsupervised human attention models in generating human-like scanpaths. Full paper available at TMLR: https://openreview.net/forum?id=7iSYW1FRWA.

Submission Type: Extended Abstract

Travel Award - Academic Status: Post-doc

Travel Award - Institution And Country: N/A

Travel Award - Low To Lower-middle Income Countries: N/A