Using Dynamic Neural Networks to Model the Speed-Accuracy Trade-Off in People

Ajay Subramanian, Omkar Kumbhar, Elena Sizikova, Najib J. Majaj, Denis Pelli

Keywords: visual object recognition, computer vision, psychophysics, psychology

TL;DR: We collect a human benchmark of the speed-accuracy trade-offs for an object recognition task and model it with neural networks

Abstract: Neural networks have been shown to exhibit remarkable object recognition performance. We ask here whether such networks can provide a useful model for how people recognize objects. Human recognition time varies, from 0.1 to 10 s, depending on the stimulus and task. Slowness of recognition is a key feature in some public health issues, such as dyslexia, so it is crucial to create a model of human speed-accuracy trade-offs. This is an essential aspect of any useful computational model of human cognitive behavior. We present a benchmark dataset for human speed-accuracy trade-off in recognizing a CIFAR-10 image~\cite{Krizhevsky09learningmultiple} from a set of provided class labels. Within a series of trials, a beep sounds at a fixed delay after the target (the desired reaction time), and the response counts only if it occurs near that time. We observe that accuracy grows with reaction time and examine several dynamic neural networks that exhibit a speed-accuracy trade-off as humans do. After limiting the network resources and adding image perturbations (grayscale conversion, noise, blur) to bring the two observers (human and network) into the same accuracy range, humans and networks show very similar dependence on duration or floating point operations (FLOPS). We conclude that dynamic neural networks are a promising model of human reaction time in recognition tasks. Understanding how the brain allocates appropriate resources under time pressure would be a milestone in neuroscience and a first step toward understanding conditions like dyslexia. Our dataset and code are publicly available.

URL: https://github.com/ajaysub110/anytime-prediction