Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems

Manuel Brenner, Florian Hess, Jonas Magdy Mikhaeil, Leonard Bereska, Zahra Monfared, Po-Chen Kuo, Daniel Durstewitz

Abstract: In many scientific disciplines, we are interested in inferring the nonlinear dynamical system underlying a set of observed time series, a challenging task in the face of chaotic behavior and noise. Pre-vious deep learning approaches toward this goal often suffered from a lack of interpretability and tractability. In particular, the high-dimensional latent spaces often required for a faithful embed-ding, even when the underlying dynamics lives on a lower-dimensional manifold, can hamper theoretical analysis. Motivated by the emerging principles of dendritic computation, we aug-ment a dynamically interpretable and mathemat-ically tractable piecewise-linear (PL) recurrent neural network (RNN) by a linear spline basis expansion. We show that this approach retains all the theoretically appealing properties of the simple PLRNN, yet boosts its capacity for approx-imating arbitrary nonlinear dynamical systems in comparatively low dimensions. We employ two frameworks for training the system, one combin-ing back-propagation-through-time (BPTT) with teacher forcing, and another based on fast and scalable variational inference. We show that the dendritically expanded PLRNN achieves better reconstructions with fewer parameters and dimen-sions on various dynamical systems benchmarks and compares favorably to other methods, while retaining a tractable and interpretable structure.