Control with Patterns: A D-learning Method

Quan Quan, Kai-Yuan Cai, Chenyu Wang

Keywords: Lyapunov Methods, Reinforcement Learning, Control with Patterns, D-learning, Visual Servoing

TL;DR: The proposed control with patterns based on D-learning is a type of Lyapunov function learning methods without requring any knowledge of the system dynamics, which parallels to Q-learning in reinforcement learning.

Abstract:

Learning-based control policies are widely used in various tasks in the field of robotics and control. However, formal (Lyapunov) stability guarantees for learning-based controllers with nonlinear dynamical systems are challenging to obtain.
We propose a novel control approach, namely Control with Patterns (CWP), to address the stability issue over data sets corresponding to nonlinear dynamical systems.
For data sets of this kind, we introduce a new definition, namely exponential attraction on data sets, to describe nonlinear dynamical systems under consideration. The problem of exponential attraction on data sets is converted to a pattern classification one based on the data sets and parameterized Lyapunov functions. Furthermore, D-learning is proposed as a method for performing CWP without knowledge of the system dynamics.
Finally, the effectiveness of CWP based on D-learning is demonstrated through simulations and real flight experiments. In these experiments, the position of the multicopter is stabilized using only real-time images as feedback, which can be considered as an Image-Based Visual Servoing (IBVS) problem.

Student Paper: no

Submission Number: 203