Does the neuronal noise in cortex help generalization?

Brian Hu, Jiaqi Shang, Ramakrishnan Iyer, Josh Siegle, Stefan Mihalas

TL;DR: We study the structure of noise in the brain and find it may help generalization by moving representations along in-class stimulus variations.

Keywords: noise, trial-to-trial variability, subspace, generalization, dropout

Abstract: Neural activity is highly variable in response to repeated stimuli. We used an open dataset, the Allen Brain Observatory, to quantify the distribution of responses to repeated natural movie presentations. A large fraction of responses are best fit by log-normal distributions or Gaussian mixtures with two components. These distributions are similar to those from units in deep neural networks with dropout. Using a separate set of electrophysiological recordings, we constructed a population coupling model as a control for state-dependent activity fluctuations and found that the model residuals also show non-Gaussian distributions. We then analyzed responses across trials from multiple sections of different movie clips and observed that the noise in cortex aligns better with in-clip versus out-of-clip stimulus variations. We argue that noise is useful for generalization when it moves along representations of different exemplars in-class, similar to the structure of cortical noise.