Dynamic Function Learning through Control of Ensemble Systems

Wei Zhang, Vignesh Narayanan, Jr-Shin Li

Keywords: Function approximation, Dynamical systems, Theory

TL;DR: We provide rigorous theoretical underpinnings of function learning through ensemble control theory.

Abstract: Learning tasks involving function approximation are prevalent in numerous domains of science and engineering. The underlying idea is to design a learning algorithm that generates a sequence of functions converging to the desired target function with arbitrary accuracy by using the available data samples. In this paper, we present a novel interpretation of iterative function learning through the lens of ensemble dynamical systems, with an emphasis on establishing the equivalence between convergence of function learning algorithms and asymptotic behavior of ensemble systems. In particular, given a set of observation data in a function learning task, we prove that the procedure of generating an approximation sequence can be represented as a steering problem of a dynamic ensemble system defined on a function space. This in turn gives rise to an ensemble systems-theoretic approach to the design of ``continuous-time" function learning algorithms, which have a great potential to reach better generalizability compared with classical ``discrete-time" learning algorithms.