Enriching ConvNets with pre-cortical processing enhances alignment with human brain responses

Published: 02 Mar 2024, Last Modified: 05 May 2024ICLR 2024 Workshop Re-Align PosterEveryoneRevisionsBibTeXCC BY 4.0
Track: long paper (up to 9 pages)
Keywords: Convolutional Neural Networks, Representational Alignment, Pre-cortical Filters
TL;DR: Differential spatial sampling enhances predictive power of convolutional neural network of human EEG data
Abstract: Convolutional Neural Networks (ConvNets) are the current state-of-the-art for modelling human visual processing whilst also performing tasks on a human per- formance level. Convolutional features can be seen as analogous to visual recep- tive fields and thus render them biologically plausible. However, spatially-uniform sampling and reuse of features across the entire visual field do not accurately rep- resent structural properties of the human visual system. Here, we present empir- ical findings of incorporating functional and structural properties of the human retina into ConvNets on their alignment with human brain activity. We show that predictions of human EEG data using ConvNets features improve by using foveated stimuli and that differential spatial sampling in ConvNets explains sev- eral qualities of EEG recordings. We also find that color and contrast filtering of inputs in turn do not yield improved predictions. Overall, our results suggest that incorporating biologically plausible spatial sampling is important for increasing representational alignment between ConvNets and humans.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Submission Number: 53
Loading