Long Short-Term Memory Neural Network Equilibria Computation and Analysis

Massinissa Amrouche, Deka Shankar Anand, Aleksandra Lekić, Vicenç Rubies Royo, Elaina Teresa Chai, Dušan M. Stipanović, Boris Murmann, Claire J. Tomlin

Keywords: Dynamical Systems, Recurrent Networks, Decision and Control, Control Theory

TL;DR: We use a homotopy formulation approach to compute the non-trivial equilibria of autonomous LSTM neural networks and numerically study the behavior of the eigenvalues of the linearized models around these nontrivial equilibria

Abstract: This paper presents a comprehensive approach for computing nontrivial equilibria of autonomous Long Short-Term Memory neural networks using a homotopy formulation. Through simulations, it is shown that the eigenvalues of the linearized models around these nontrivial equilibria tend to move closer to the unit circle as the complexity of the training data increases. This provides insights into the dynamical properties of the LSTM neural networks.