Federated Learning on Adaptively Weighted Nodes by Bilevel Optimization

Yankun Huang, Qihang Lin, Nick Street, Stephen Baek

Keywords: federated learning, bilevel optimization, distributed optimization, generalization performance

TL;DR: We propose a federated learning method with adaptively weighted nodes and analyze its generalization performance.

Abstract: We propose a federated learning method with weighted nodes in which the weights can be modified to optimize the model’s performance on a separate validation set. The problem is formulated as a bilevel optimization problem where the inner problem is a federated learning problem with weighted nodes and the outer problem focuses on optimizing the weights based on the validation performance of the model returned from the inner problem. A communication-efficient federated optimization algorithm is designed to solve this bilevel optimization problem. We analyze the generalization performance of the output model and identify the scenarios when our method is in theory superior to training a model locally and superior to federated learning with static and evenly distributed weights.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: General Machine Learning (ie none of the above)