A theory of parameter identifiability in data-constrained recurrent neural networks

ICLR 2026 Conference Submission10663 Authors

Keywords: computational neuroscience, recurrent neural networks, identifiability

TL;DR: A theory of parameter identifiability in data-constrained recurrent neural networks

Abstract: An increasingly common approach in neuroscience seeks to understand the brain by training recurrent neural networks (RNNs) to reproduce observed neural activity. Unlike brains, these RNNs can be computationally poked and perturbed to reveal principles central to their function. However, whether the insights gained from these RNNs truly apply to biological neural circuits remains an open question. The answer hinges on a key distinction: which RNN parameters are uniquely determined by the data they are trained on (i.e., identifiable), and which are unconstrained? To this end, we develop a framework that isolates identifiable subspaces of the RNN parameters, leading to several key findings: (i) commonly used RNN estimators have unconstrained parameters and the dimensionality of training data, i.e., the trajectories in neural state space, dictates the extent of parameter constraints; (ii) we can design RNN estimators to remain confined to identifiable components; (iii) we propose intervention experiments to expand the identifiable subspace; and (iv) we prove that changes in non-identifiable components preserve dynamics on identifiable subspaces but can introduce spurious structure elsewhere. Together, these results delineate regions of state space where RNN predictions are reliable and pinpoint where they are not. Our theory shows that current RNNs are not valid proxies of neural circuits, as their predictions and interpretation can be swayed by non-identifiable components. These results define guidelines for the responsible use of RNN models in neuroscience.

Primary Area: applications to neuroscience & cognitive science

Submission Number: 10663