PARS-Push: Personalized, Asynchronous and Robust Decentralized OptimizationDownload PDF

26 May 2022 (modified: 05 May 2023)ICML 2022 Pre-training WorkshopReaders: Everyone
Keywords: Model-Agnostic Meta-Learning, Asynchronous Communication, Decentralized Optimization, Personalization
Abstract: We study the decentralized multi-step Model-Agnostic Meta-Learning (MAML) framework where a group of $n$ agents seeks to find a common point that enables ``few-shot'' learning (personalization) via local stochastic gradient steps on their local functions. We formulate the personalized optimization problem under the MAML framework and propose PARS-Push, a decentralized asynchronous algorithm robust to message failures, communication delays, and directed message sharing. We characterize the convergence rate of PARS-Push for smooth and strongly convex and smooth and non-convex functions under arbitrary multi-step personalization. Moreover, we provide numerical experiments showing its performance under heterogeneous data setups.
0 Replies