Online Optimization of Closed-Loop Control Systems

Hao Ma, Melanie Zeilinger, Michael Muehlebach

Keywords: Gradient-Based Stochastic Optimization, Online Learning, Non-Convex Optimization, Reinforcement Learning, Cyber-Physical Systems, Robotic Control

Abstract: We propose a novel gradient-based online optimization framework for solving stochastic programming problems that frequently arise in the context of cyber-physical and robotic systems. We establish the connection between our algorithms and the cyber-physical systems through the classic two-degree-of-freedom control loop. We also incorporate an approximate model of the dynamics as prior knowledge into the learning process, and characterize the impact of modeling errors in the system dynamics on the convergence rate of the algorithms. We show that even rough estimates of the dynamics can significantly improve the convergence of our algorithms. Finally, we evaluate our algorithms in simulations of a flexible beam and a four-legged walking robot.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Hao_Ma6

Track: Regular Track: unpublished work

Submission Number: 53