Reinforcement Learning for Control with Stability Guarantee
Primary Area: reinforcement learning
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Reinforcement learning, Lyapunov stable dynamic model, uniformly ultimately bounded, iterative learning framework
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Reinforcement learning (RL) has achieved promising performance in complicated system control.
To overcome this defect, we firstly apply a Lyapunov stable dynamical model as a reference system to fit the real system. Then, we prove that if the state fitting error between the reference and real system are bounded, the real system has Uniformly Ultimately Bounded (UUB) stability guarantee. Motivated by our theoretical analysis, we guide the design of reward functions for RL based on conditions of UUB guarantee for real systems and propose $\textbf{ITSRL}$, an $\textbf{I}$terative $\textbf{T}$raining framework for learning $\textbf{S}$table $\textbf{RL}$ control policy with UUB stability guarantee, by iteratively minimizing the state fitting error between the reference and real system, which can be adapted to various advanced RL methods. Our evaluation results on three control tasks demonstrate that the proposed ITSRL framework can improve the performance of RL controller under perturbation.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 2293
Loading