Learning Control Lyapunov Functions For High-dimensional Unknown Systems using Guided Iterative State Space Exploration

Songyuan Zhang, Chuchu Fan

Keywords: Neural Certificates, Lyapunov Functions, Robotics

TL;DR: We develop a novel algorithm that learns a stable controller for high-dimensional unknown systems. We provide theoretical guarantees for the convergence, and empirical results that it outperforms other baselines in a suite of environments.

Abstract: Designing stable controllers in complex, high-dimensional systems with unknown dynamics is a critical problem when we deploy robots in the real world. Prior works use learning-based control Lyapunov functions (CLFs) or adaptive control to derive such controllers, but they suffer from two significant challenges: scalability and model transparency. This paper proposes a general framework to jointly learn the local dynamics, a stable controller, and the corresponding CLF in high-dimensional unknown systems. Our approach, GIE-CLF, does not need any knowledge of the environment, such as the dynamics, reward functions, etc, and can scale up to high dimensional systems using only local knowledge of the dynamics inside a trusted tunnel instead of global knowledge required by other methods. We provide theoretical guarantees for our framework and demonstrate it on highly complex systems including a high-fidelity F-16 jet aircraft model that has a 16-dimensional state space and a 4-dimensional input space. Experimental results show that GIE-CLF significantly outperforms prior works in reinforcement learning and imitation learning. We also show that our algorithm can also be extended to learn other control certificate functions for unknown systems.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: General Machine Learning (ie none of the above)