Deep neural network model of sound localization replicates “what” and “where” representations in auditory cortex

Chenggang Chen, Zhiyu Yang, Xiaoqin Wang

Keywords: NeuroAI, Auditory Cortex, Sound Localization, Computational Neuroscience, Dual Pathway

TL;DR: Auditory “what” and “where” in AI

Abstract: Unlike visual cortex, whether auditory cortex has parallel pathways for sound identification (“what”) and localization (“where”) is debated. It also lacks a topographic map of auditory space, like the retinotopy in visual cortex. Here, we examined auditory “what” and “where” representations in deep neural network models of sound localization. Surprisingly, the models learned well-separated clusters by sound type, but not by sound level. Sounds were further organized by spectrogram similarity, and these organizations were aligned with human spatial hearing. Sound type also determined whether the representations of sound locations organized into a map: maps were formed in both the horizontal and vertical planes when sounds contained binaural and monaural localization cues that were topographically organized relative to human ears. However, formation of a spatial map did not improve, but rather deteriorated, both the model’s and humans’ localization accuracy. Together, our model suggests that “what” cannot be dissociated from “where” in auditory cortex. A space map is created by spatially organized localization cues and is unnecessary for auditory cortex.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 4164