Natural gradient enables fast sampling in spiking neural networksDownload PDF

Published: 31 Oct 2022, Last Modified: 09 Jan 2023NeurIPS 2022 AcceptReaders: Everyone
Keywords: spiking neural networks, sampling, Bayesian inference, information geometry, neural population geometry
TL;DR: We derive a unified framework for spiking neural networks that can sample at behaviorally-relevant timescales through population geometry.
Abstract: For animals to navigate an uncertain world, their brains need to estimate uncertainty at the timescales of sensations and actions. Sampling-based algorithms afford a theoretically-grounded framework for probabilistic inference in neural circuits, but it remains unknown how one can implement fast sampling algorithms in biologically-plausible spiking networks. Here, we propose to leverage the population geometry, controlled by the neural code and the neural dynamics, to implement fast samplers in spiking neural networks. We first show that two classes of spiking samplers---efficient balanced spiking networks that simulate Langevin sampling, and networks with probabilistic spike rules that implement Metropolis-Hastings sampling---can be unified within a common framework. We then show that careful choice of population geometry, corresponding to the natural space of parameters, enables rapid inference of parameters drawn from strongly-correlated high-dimensional distributions in both networks. Our results suggest design principles for algorithms for sampling-based probabilistic inference in spiking neural networks, yielding potential inspiration for neuromorphic computing and testable predictions for neurobiology.
Supplementary Material: pdf
25 Replies