Enriching ConvNets with pre-cortical processing enhances alignment with human brain responses

Niklas Müller, H.Steven Scholte, Iris Groen

Track: long paper (up to 9 pages)

Keywords: Convolutional Neural Networks, Representational Alignment, Pre-cortical Filters

TL;DR: Differential spatial sampling enhances predictive power of convolutional neural network of human EEG data

Abstract:

Convolutional Neural Networks (ConvNets) are the current state-of-the-art for
modelling human visual processing whilst also performing tasks on a human per-
formance level. Convolutional features can be seen as analogous to visual recep-
tive fields and thus render them biologically plausible. However, spatially-uniform
sampling and reuse of features across the entire visual field do not accurately rep-
resent structural properties of the human visual system. Here, we present empir-
ical findings of incorporating functional and structural properties of the human
retina into ConvNets on their alignment with human brain activity. We show
that predictions of human EEG data using ConvNets features improve by using
foveated stimuli and that differential spatial sampling in ConvNets explains sev-
eral qualities of EEG recordings. We also find that color and contrast filtering of
inputs in turn do not yield improved predictions. Overall, our results suggest that
incorporating biologically plausible spatial sampling is important for increasing
representational alignment between ConvNets and humans.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 53