Benchmarking human visual search computational models in natural scenes: models comparison and reference datasets

Fermín Travi, Gonzalo Ruarte, Gaston Bujia, Juan E Kamienkowski

Keywords: visual search, eye movements, computational models, ideal bayesian observer

TL;DR: The present work compares state-of-the-art visual search models in different datasets comprising eye movements in natural scenes and sheds light on their limitations and how potential improvements could be accomplished.

Abstract:

Visual search is an essential part of almost any everyday human goal-directed interaction with the environment.
Nowadays, several algorithms are able to predict gaze positions during simple observation, but few models attempt to simulate human behavior during visual search in natural scenes. Furthermore, these models vary widely in their design and exhibit differences in the datasets and metrics with which they were evaluated. Thus, there is a need for a reference point, on which each model can be tested and from where potential improvements can be derived. In the present work, we select publicly available state-of-the-art visual search models in natural scenes and evaluate them on different datasets, employing the same metrics to estimate their efficiency and similarity with human subjects. In particular, we propose an improvement to the Ideal Bayesian Searcher through a combination with a neural network-based visual search model, enabling it to generalize to other datasets.
The present work sheds light on the limitations of current models and how potential improvements can be accomplished by combining approaches. Moreover, it moves forward on providing a solution for the urgent need for benchmarking data and metrics to support the development of more general human visual search computational models.

Community Implementations: [ 6 code implementations](https://www.catalyzex.com/paper/benchmarking-human-visual-search/code)