Enriching ConvNets with pre-cortical processing enhances alignment with human brain responses
Track: long paper (up to 9 pages)
Keywords: Convolutional Neural Networks, Representational Alignment, Pre-cortical Filters
TL;DR: Differential spatial sampling enhances predictive power of convolutional neural network of human EEG data
Abstract: Convolutional Neural Networks (ConvNets) are the current state-of-the-art for
modelling human visual processing whilst also performing tasks on a human per-
formance level. Convolutional features can be seen as analogous to visual recep-
tive fields and thus render them biologically plausible. However, spatially-uniform
sampling and reuse of features across the entire visual field do not accurately rep-
resent structural properties of the human visual system. Here, we present empir-
ical findings of incorporating functional and structural properties of the human
retina into ConvNets on their alignment with human brain activity. We show
that predictions of human EEG data using ConvNets features improve by using
foveated stimuli and that differential spatial sampling in ConvNets explains sev-
eral qualities of EEG recordings. We also find that color and contrast filtering of
inputs in turn do not yield improved predictions. Overall, our results suggest that
incorporating biologically plausible spatial sampling is important for increasing
representational alignment between ConvNets and humans.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Submission Number: 53
Loading