Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors

Finn Schmidt, Polina Turishcheva, Suhas Shrinivasan, Fabian H. Sinz

Keywords: Dynamics of neural representations, probabilistic neural predictive models, primary visual cortex of mice, variational encoding models, stimulus-independent variability

TL;DR: This paper presents a probabilistic model combining video inputs with stimulus-independent latent factors to model dynamic neural activity in mouse V1.

Abstract:

The neural activity in the visual processing is influenced by both external stimuli and internal brain states.
Ideally, a neural predictive model should account for both of them.
Currently, there are no dynamic encoding models that explicitly model a latent state and the entire neuronal response distribution.
We address this gap by proposing a probabilistic model that predicts the joint distribution of the neuronal responses from video stimuli and stimulus-independent latent factors.
After training and testing our model on mouse V1 neuronal responses, we find that it outperforms video-only models in terms of log-likelihood and achieves improvements in likelihood and correlation when conditioned on responses from other neurons.
Furthermore, we find that the learned latent factors strongly correlate with mouse behavior and that they exhibit patterns related to the neurons’ position on the visual cortex, although the model was trained without behavior and cortical coordinates.
Our findings demonstrate that unsupervised learning of latent factors from population responses can reveal biologically meaningful structure that bridges sensory processing and behavior, without requiring explicit behavioral annotations during training.
The code is attached to the submission.

Primary Area: Neuroscience and cognitive science (e.g., neural coding, brain-computer interfaces)

Submission Number: 21358