Lurie Networks with Robust k-contracting Dynamics
Abstract: The Lurie network is a novel and unifying time-invariant neural ODE. Many existing continuous-time models, including recurrent neural networks and neural oscillators, are special cases of the Lurie network in this context. Mild constraints on the weights and biases of the Lurie network are derived to ensure a generalised concept of stability is guaranteed. This generalised stability measure is that of k-contraction which permits global convergence to a point, line or plane in the neural state-space. This includes global convergence to one of multiple equilibrium points or limit cycles as observed in many dynamical systems including associative and working memory. Weights and biases of the Lurie network, which satisfy the k-contraction constraints, are encoded through unconstrained parametrisations. The novel stability results and parametrisations provide a toolset for training over the space of k-contracting Lurie network's using standard optimisation algorithms. These results are also leveraged to construct and train a graph Lurie network satisfying the same convergence properties. Empirical results show the improvement in prediction accuracy, generalisation and robustness on a range of simulated dynamical systems, when the graph structure and k-contraction conditions are introduced. These results also compare favourably against other well known stability-constrained models and an unconstrained neural ODE.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: All major changes are detailed in response to reviewers comments. Code has been removed as supplementary material as the link to the GitHub repo is now included in the paper.
Code: https://github.com/CR-Richardson/LurieNetwork
Assigned Action Editor: ~Yoshinobu_Kawahara1
Submission Number: 3842
Loading