Once-for-All Federated Learning: Learning From and Deploying to Heterogeneous Clients

Kamala Varma, Enmao Diao, Tanya Roosta, Jie Ding, Tao Zhang

Keywords: Federated Learning, Internet of Things, Heterogeneity

TL;DR: A federated learning method that supports the efficient deployment of sub-networks that cater to client-specific device resource constraints.

Abstract: Federated learning (FL) enables multiple client devices to train a single machine learning model collaboratively. As FL often involves various smart devices, it is important to adapt the FL pipeline to accommodate device resource constraints. This work addresses the problem of training and storing memory-intensive deep neural network architectures on resource-constrained devices. Existing solutions often involve computationally expensive methods. We propose Once-for-All Federated Learning (OFA-FL) to overcome this limitation by learning a model that concurrently optimizes sub-networks of various sizes. Clients can therefore receive the sub-network best suited for their device resources without extra computation. Our experiments show that each component of OFA-FL contributes to well-performing FL-produced sub-networks while maintaining a global network design that supports the efficient deployment of device resource-specific sub-networks.