How Few Davids Improve One Goliath: Federated Learning in Resource-Skew Edge Computing Environments

Jiayun Zhang, Shuheng Li, Haiyu Huang, Zihan Wang, Xiaohan Fu, Dezhi Hong, Rajesh K. Gupta, Jingbo Shang

Keywords: federated learning, edge computing, resource skewness

Abstract:

The real-world deployment of federated learning requires orchestrating clients with widely varying compute resources, from strong enterprise-level devices in data centers to weak mobile and Web-of-Things (WoT) devices. Prior works have explored ways to downscale large models for weak devices and perform aggregation among heterogeneous models. A typical architectural assumption is that there are equally many strong and weak devices. In reality, we often see resource-skew environments where a few (1 or 2) strong devices hold a substantial amount of data resources, accompanied by a large number of weak devices. This poses challenges—the unshared portion of the large model rarely receives updates or derives benefits from the weak collaborators.

We aim to facilitate reciprocal benefits between strong and weak devices in the presence of resource skewness in federated learning. We propose RecipFL, a novel framework featuring a server-side graph hypernetwork that generates weights for personalized client models, aligning them with the unique data distributions and computational capacities of individual devices. The graph hypernetwork captures local and global structures of client models and generalizes knowledge about model weights across model architectures. Notably, RecipFL is agnostic to model scaling strategies and can enable collaboration among arbitrary neural network models. We establish the generalization bound of RecipFL through theoretical analysis and conduct extensive experiments across image classification and natural language inference tasks with various model architectures. The results show that RecipFL can improve accuracy by 3.6% and 8.7% on strong and weak devices respectively, providing incentives for strong devices to actively participate in federated learning.

Track: Systems and Infrastructure for Web, Mobile, and WoT

Submission Guidelines Scope: Yes

Submission Guidelines Blind: Yes

Submission Guidelines Format: Yes

Submission Guidelines Limit: Yes

Submission Guidelines Authorship: Yes

Student Author: Yes

Submission Number: 1344