Provable Regret Bounds for Deep Online Learning and Control

Xinyi Chen, Edgar Minasyan, Jason D. Lee, Elad Hazan

Keywords: online learning, deep neural networks, online control, nonstochastic control

Abstract:

The use of deep neural networks has been highly successful in reinforcement learning and control,
although few theoretical guarantees for deep learning exist for these problems.
There are two main challenges for deriving performance guarantees: a) control has state information and thus is inherently online and b) deep networks are non-convex predictors for which online learning cannot provide provable guarantees in general.

Building on the linearization technique for overparameterized neural networks, we derive provable regret bounds for efficient online learning with deep neural networks. Specifically, we show that over any sequence of convex loss functions, any low-regret algorithm can be adapted to optimize the parameters of a neural network such that it competes with the best net in hindsight. As an application of these results in the online setting, we obtain provable bounds for online episodic control with deep neural network controllers.