Modeling Human Eye Movements with Neural Networks in a Maze-Solving TaskDownload PDF

Published: 20 Oct 2022, Last Modified: 22 Oct 2023Gaze Meets ML 2022 OralReaders: Everyone
Keywords: Saccade, Recurrent Network, Maze, Psychophysics, Fovea, Mental Simulation
TL;DR: A recurrent neural network with novel fovea architecture trained on a maze-solving task produces human-like saccades.
Abstract: From smoothly pursuing moving objects to rapidly shifting gazes during visual search, humans employ a wide variety of eye movement strategies in different contexts. While eye movements provide a rich window into mental processes, building generative models of eye movements is notoriously difficult, and to date the computational objectives guiding eye movements remain largely a mystery. In this work, we tackled these problems in the context of a canonical spatial planning task, maze-solving. We collected eye movement data from human subjects and built deep generative models of eye movements using a novel differentiable architecture for gaze fixations and gaze shifts. We found that human eye movements are best predicted by a model that is optimized not to perform the task as efficiently as possible but instead to run an internal simulation of an object traversing the maze. This not only provides a generative model of eye movements in this task but also suggests a computational theory for how humans solve the task, namely that humans use mental simulation.
Submission Type: Full Paper
Travel Award - Academic Status: Undergraduate
Travel Award - Institution And Country: Massachusetts Institute of Technology, USA
Travel Award - Low To Lower-middle Income Countries: No, my institution does not qualify.
Camera Ready Latexfile: zip
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 1 code implementation](
4 Replies