Adaptive optimal control of unknown discrete-time linear systems with guaranteed prescribed degree of stability using reinforcement learning

Seyed Ehsan Razavi, Mohammadamin Moradi, Saeed Shamaghdari, Mohammad Bagher Menhaj

Abstract: This paper proposes a model-free solution for solving the optimal regulation problem for a discrete-time linear time-invariant system that unlike previous methods, presents a guaranteed convergence rate of the state variables as is needed in a group of problems. Initially, the Linear Quadratic Regulation problem (LQR) with a guaranteed convergence rate of the state is formulated for a system with known dynamics and the associated Riccati equation is derived. Solving the Riccati equation and finding the state feedback gain requires full knowledge of the dynamics of the system. To overcome this problem, the Policy Iteration (PI) Reinforcement Learning (RL) algorithm is formulated to solve the LQR problem with a guaranteed convergence rate, and the optimal state feedback gain is derived without having any knowledge about the dynamics of the system and only through the measurement of the states of the system. Eventually, the validity of the results is shown through simulation.