Common visual learning constraints in transformers and newborn brains: Evidence from line drawings

Lalit Pandey, Samantha Marie Waters Wood, Justin Newell Wood

Keywords: vision transformers, convolutional neural networks, newborns, chicks, object recognition, line drawings, temporal learning

TL;DR: By running parallel controlled-rearing experiments on newborn chicks and deep neural networks, we found that transformers and newborn brains have common learning constraints.

Abstract: A core goal in artificial intelligence (AI) is to build machines that learn like brains. Many AI systems, including convolutional neural networks (CNNs) and vision transformers (ViTs), rival human adults on visual recognition tasks. But, do these AI systems actually learn like brains? If so, AI systems should produce the same learning outcomes as brains when trained with the same data. Here, we tested whether AI systems learn the same object recognition skills as newborn chicks when trained in the same visual environments as chicks. We performed digital twin studies of prior controlled-rearing experiments, evaluating whether CNNs and ViTs produce the same pattern of successes and failures as chicks. When ViTs were equipped with a biologically inspired temporal learning objective, the ViTs showed the same learning patterns as chicks: both learned object recognition when reared with normal objects, but failed to learn object recognition when reared with line drawings. Conversely, when CNNs were equipped with the same temporal learning objective, the CNNs showed a different pattern from chicks: CNNs learned object recognition whether exposed to normal objects or line drawings. These results show that transformers can be accurate image-computable models of visual learning.

Submission Number: 15