Learning nonlinear dynamical systems from a single trajectory

Dylan Foster, Tuhin Sarkar, Alexander Rakhlin

Abstract: We introduce algorithms for learning nonlinear dynamical systems of theform $x_{t+1}=\sigma(\Theta{}x_t)+\varepsilon_t$, where $\Theta$ is a weightmatrix, $\sigma$ is a nonlinear monotonic link function, and$\varepsilon_t$ is a mean-zero noise process. When the link function is known, wegive an algorithm that recovers the weight matrix $\Theta$ from a single trajectorywith optimal sample complexity and linear running time. The algorithmsucceeds under weaker statistical assumptions than in previous work, and inparticular i) does not require a bound on the spectral norm of the weightmatrix $\Theta$ (rather, it depends on a generalization of thespectral radius) and ii) works when the link function is the ReLU. Our analysis has three keycomponents: i) We show how \emph{sequential Rademacher complexities} can beused to provide generalization guarantees for general dynamicalsystems, ii) we give a general recipe whereby global stability fornonlinear dynamical systems can be used to certify that the state-vector covariance is well-conditioned, and iii) using these tools, we extend well-known algorithms for efficiently learning generalized linear models to the dependent setting.