Immediate generalisation in humans but a generalisation lag in deep neural networks—evidence for representational divergence?
Track: long paper (up to 9 pages)
Keywords: representation learning, object recognition, computer vision, deep neural networks, representational alignment, psychophysics, behavioral comparison, human-machine comparison
TL;DR: An empirical side-by-side comparison of supervised representation learning in humans and deep neural networks revealed that, although DNNs are not inherently less data-efficient, they exhibit a pronounced generalisation lag.
Abstract: Recent research has seen many behavioral comparisons between humans and deep neural networks (DNNs) in the domain of image classification. Often, comparison studies focus on the end-result of the learning process by measuring and comparing the similarities in the representations of object categories once they have been formed. However, the process of how these representations emerge—that is, the behavioral changes and intermediate stages observed during the acquisition—is less often directly and empirically compared.
Here we report a detailed investigation of how transferable representations are acquired in human observers and various classic and state-of-the-art DNNs. We develop a constrained supervised learning environment in which we align learning-relevant parameters such as starting point, input modality, available input data and the feedback provided. Across the whole learning process we evaluate and compare how well learned representations can be generalized to previously unseen test data.
Our findings indicate that in terms of absolute classification performance DNNs demonstrate a level of data efficiency comparable to—and sometimes even exceeding that—of human learners, challenging some prevailing assumptions in the field. However, comparisons across the entire learning process reveal significant representational differences: while DNNs' learning is characterized by a pronounced generalisation lag, humans appear to immediately acquire generalizable representations without a preliminary phase of learning training set-specific information that is only later transferred to novel data.
Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.
Submission Number: 41
Loading