Uncovering Semantic Selectivity of Latent Groups in Higher Visual Cortex with Mutual Information-Guided Diffusion

Yule Wang, Joseph Yu, Chengrui Li, Weihan Li, Anqi Wu

Keywords: Neural Latent Discovery, Diffusion Models, Visual Cortex

TL;DR: We propose MIG-Vis, a method that uses mutual information-guided diffusion to synthesize images, thereby uncovering neural latent groups that exhibit clear semantic selectivity for pose, inter-category variation, and intra-category content.

Abstract: Understanding how neural populations in higher visual areas encode object-centered visual information remains a key challenge in computational neuroscience. Prior work has examined representational alignment between artificial neural networks and the visual cortex, but these findings are indirect and provide limited insight into the structure of neural population coding. Decoding-based methods can recover semantic features from neural activity, yet they do not reveal how these features are organized. This leaves an open question: how feature-specific visual information is distributed across neural populations, and whether it forms structured, semantically meaningful subspaces. To address this, we introduce MIG-Vis, a method that leverages diffusion models to visualize and validate visual-semantic attributes encoded in neural latent subspaces. MIG-Vis first learns a group-wise disentangled neural latent representation using a variational autoencoder, and then applies mutual information (MI)-guided diffusion synthesis to visualize the visual-semantic features encoded in each latent group. We validate MIG-Vis on multi-session neural spiking datasets from the inferior temporal (IT) cortex of two macaques. The synthesized results show that MIG-Vis identifies neural latent groups with clear semantic selectivity to diverse visual features, including object pose, inter-category variation, and intra-category content. These findings provide direct and interpretable evidence of structured semantic representation in higher visual cortex.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 2673