Continuous-time neural networks for modeling linear dynamical systems

Chinmay Datar, Adwait Datar, Felix Dietrich, Wil Schilders

Keywords: Continuous-time neural networks, Neural Architecture Search (NAS), Ordinary Differential Equations (ODEs), error analysis

TL;DR: We propose a gradient-free and numerically stable algorithm to compute architecture and parameters of our neural networks for modeling linear dynamical systems.

Abstract: We propose to model Linear Time-Invariant (LTI) systems as a first step towards constructing sparse neural networks for modeling more complex dynamical systems. We use a variant of continuous-time neural networks in which the output of each neuron evolves continuously as a solution of a first or second-order Ordinary Differential Equation (ODE). Instead of computing the network parameters from data, we rely on system identification techniques to obtain a state-space model. Our algorithm is gradient-free, numerically stable, and computes a sparse architecture together with all network parameters from the given state-space matrices of the LTI system. We provide an upper bound on the numerical errors for our constructed neural networks and demonstrate their accuracy by simulating the transient convection-diffusion equation.

Submission Number: 61