Decentralized Online Nonparametric Learning

Alec Koppel, Santiago Paternain, Cédric Richard, Alejandro Ribeiro

Abstract: We consider decentralized online supervised learning where estimators are chosen from a reproducing kernel Hilbert space (RKHS). Here a multi-agent network aims to learn nonlinear statistical models that are optimal in terms of a global convex functional that aggregates data across the network, while only having access to locally observed streaming data. We address this problem by allowing each agent to learn a local copy of the global regression function while enforcing consensus constraints. We use a penalized variant of functional stochastic gradient descent operating simultaneously with low-dimensional subspace projections. The resulting algorithm allows each individual agent to learn, based upon its locally observed data stream and message passing with its neighbors, a function that is provably close to globally optimal and satisfies the consensus constraints. Moreover, the complexity of the learned regression functions is guaranteed to be finite. We then validate this approach on the Brodatz textures dataset for the case of decentralized online multi-class kernel logistic regression.