Human-like multiple object tracking through occlusion via gaze-following

Benjamin Peters, Eivinas Butkus, Nikolaus Kriegeskorte

Submission Type: Full Paper

Keywords: human dynamic object vision, multiple object tracking, visual occlusion, eye movements, computational modelling

TL;DR: A multi object tracking model behaved more human-like when we equipped it with a fovea and made it follow human gaze trajectories.

Abstract: State-of-the-art multiple object tracking (MOT) models have recently been shown to behave in qualitatively different ways from human observers. They exhibit superhuman performance for large numbers of targets and subhuman performance when targets disappear behind occluders. Here we investigate whether human gaze behavior can help explain differences in human and model behavior. Human subjects watched scenes with objects of various appearances. They tracked a designated subset of the objects, which moved continuously and frequently disappeared behind static black-bar occluders, reporting the designated objects at the end of each trial. We measured eye movements during tracking and tracking accuracy. We found that human gaze behavior is clearly guided by task relevance: designated objects were preferentially fixated. We compared human performance to that of cognitive models inspired by state-of-the-art MOT models with object slots, where each slot represents the model's probabilistic belief about the location and appearance of one object. In our model, incoming observations are unambiguously assigned to slots using the Hungarian algorithm. Locations are tracked probabilistically (given the hard assignment) with one Kalman filter per slot. We equipped the computational models with a fovea, yielding high-precision observations at the center and low-precision observations in the periphery. We found that constraining models to follow the same gaze behavior as humans (imposing the human-measured fixation sequences) best captures human behavioral phenomena. These results demonstrate the importance of gaze behavior, allowing the human visual system to optimally use its limited resources.

Submission Number: 29