Optimizing Server-side Aggregation For Robust Federated Learning via Subspace Training

Yueqi XIE, WEIZHONG ZHANG, Renjie Pi, Fangzhao Wu, Qifeng Chen, Tong Zhang, Xing Xie, Sunghun Kim

Keywords: federated learning, server-side aggregation, subspace training

Abstract: Non-IID data distribution across clients and poisoning attacks are two main challenges in real-world federated learning systems. While both of them have attracted great research interest with specific strategies developed, no known solution manages to address them in a unified framework. To overcome both challenges, we propose SmartFL, a generic approach that optimizes the server-side aggregation process with a small amount of on-server proxy data (e.g., around one hundred samples for CIFAR-10) via a subspace training technique. Specifically, the aggregation weight of each participating client at each round is optimized using the server-side proxy data, which is essentially the optimization of the global model in the convex hull spanned by client models. Since at each round, the number of tunable parameters optimized on the server side equals the number of participating clients (thus independent of the model size), we are able to train a global model with massive parameters using only a small amount of server-side proxy data. We provide theoretical analyses of the convergence and generalization capacity for SmartFL. Empirically, SmartFL achieves state-of-the-art performance on both federated learning with non-IID data distribution and federated learning with malicious clients.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Social Aspects of Machine Learning (eg, AI safety, fairness, privacy, interpretability, human-AI interaction, ethics)