Connections Between Optimization in Machine Learning and Adaptive Control

Anonymous

Keywords: optimization for machine learning, adaptive control, adaptive identification, online learning

Abstract: This paper explores many immediate connections between adaptive control and machine learning, both through common update laws as well as common concepts. Adaptive control as a field has focused on mathematical rigor and guaranteed convergence. The rapid advances in machine learning on the other hand have brought about a plethora of new techniques and problems for learning. This paper elucidates many of the numerous common connections between both fields such that results from both may be leveraged together to solve new problems. In particular, a specific problem related to higher order learning is solved through insights obtained from these intersections.

TL;DR: History of parallel developments in update laws and concepts between adaptive control and optimization in machine learning.