Remarks on the Gradient Training of Linear Neural Network Based Feedback for the LQR Problem

Arthur Castello Branco de Oliveira, Milad Siami, Eduardo Sontag

Abstract: Motivated by the growing use of Artificial intelligence (AI) tools in control design, this paper takes steps toward bridging results from Direct Gradient methods for the Linear Quadratic Regulator (LQR), and neural networks. More specifically, it looks into the case where one wants to find a Linear Feed-Forward Neural Network (LFFNN) feedback that minimizes the LQR cost. This work develops gradient formulas that can be used to implement the training of such networks and derives an important conservation law of the system. This conservation law is then leveraged to prove the global convergence of solutions and invariance of the set of stabilizing networks under the training dynamics. These theoretical results are followed by an extensive analysis of the simplest version of the problem (the “scalar case”) and by numerical evidence of faster convergence of the training of general LFFNNs when compared to traditional direct gradient methods.