Expressive dynamics models with nonlinear injective readouts enable reliable recovery of latent features from neural activity

Christopher Versteeg, Andrew R Sedler, Jonathan David McCart, Chethan Pandarinath

Keywords: computational neuroscience, systems neuroscience, neural population dynamics, interpretability, neuroscience, latent dynamics, neural manifolds, neural ODEs, RNNs, sequential autoencoders, invertible neural networks

TL;DR: By combining expressive dynamics models and injective readouts, we show that our new model can recover dynamical features and neural manifolds from synthetic spiking data.

Abstract: The advent of large-scale neural recordings has enabled new approaches that aim to discover the computational mechanisms of neural circuits by understanding the rules that govern how their state evolves over time. While these \textit{neural dynamics} cannot be directly measured, they can typically be approximated by low-dimensional models in a latent space. How these models represent the mapping from latent space to neural space can affect the interpretability of the latent representation. Typical choices for this mapping (e.g., linear layer or MLP) lack the property of injectivity, meaning that changes in latent state may have no effect on neural activity. During training, non-injective readouts incentivize the invention of dynamics that misrepresent the underlying system and the computation it performs. Combining our injective Flow readout with prior work on interpretable latent dynamics models, we created the Ordinary Differential equations autoencoder with Injective Nonlinear readout (ODIN), which learns to capture latent dynamical systems that are nonlinearly embedded into observed neural firing rates via an approximately injective nonlinear mapping. We show that ODIN can recover nonlinearly embedded systems from simulated neural activity, even when the nature of the system and embedding are unknown. Additionally, we show that ODIN enables the unsupervised recovery of underlying dynamical features (e.g., fixed-points) and embedding geometry. When applied to biological neural recordings, ODIN can reconstruct neural activity with comparable accuracy to previous state-of-the-art methods while using substantially fewer latent dimensions. Overall, ODIN's accuracy in recovering ground-truth latent features and ability to accurately reconstruct neural activity with low dimensionality make it a promising method for distilling interpretable dynamics that can help explain neural computation.

Submission Number: 14224