Safe Learning and Control using Meta-Learning

Thomas Lew, Apoorva Sharma, James Harrison, Marco Pavone

Keywords: Meta-Learning, Safe Learning, Optimal Control, Sequential Convex Programming, Chance-Constrained Programming, Bayesian Regression

Abstract:

When deploying autonomous systems in uncertain environments, mismatch between a model of the system dynamics and the true dynamics is inevitable. For an autonomous agent to perform tasks in such settings, it must control a system subject to model uncertainty under external disturbances, and do so safely. Furthermore, this initial uncertainty might be too high to carry out the desired task safely, in which case autonomous agents must be able to learn, using data observed online to reduce the uncertainty about the system dynamics. Recent work in meta-learning has emerged as a promising, data-driven alternative for online learning, using an offline
training phase to imbue learning algorithms with prior knowledge needed to efficiently fit data observed online. Such approaches showed to outperform nonparametric kernel-based Gaussian processes in accuracy and adaptation capabilities.

In this work, we present a unified framework for safe learning and adaptive control of an uncertain nonlinear system which leverages the computational and data efficiency gains of a meta-learned dynamics model. This framework combines three key technical contributions:
(1) Lipschitz normalization techniques to improve the uncertainty propagation properties of the meta-learned dynamics model;
(2) a tractable optimization objective for the exploration phase;
(3) formulations of exploration and exploitation tasks as chance-constrained optimal control problems, which are solved using a novel sequential convex programming algorithm.
In spite of an initially highly uncertain prior model causing the initial problem to be unfeasible, each phase is solved successively until the end region is safely reached.