FedLEGO: Enabling Heterogenous Model Cooperation via Brick Reassembly in Federated Learning

Jiaqi Wang, Suhan Cui, Fenglong Ma

Keywords: Federated Learning

TL;DR: Enabling Heterogenous Model Cooperation via Brick Reassembly in Federated Learning

Abstract: This paper focuses on addressing the practical yet challenging problem of model heterogeneity in federated learning, where clients possess models with different network structures. To track this problem, we propose a novel framework called FedLEGO, which treats each client model as a LEGO toy, reassembles it into bricks, and assembles bricks back into personalized models accordingly. Moreover, FedLEGO automatically and dynamically generates informative and diverse personalized candidates with minimal human intervention. Furthermore, our proposed heterogeneous model reassembly technique mitigates the adverse impact introduced by using public data with different distributions from the client data to a certain extent. Experimental results demonstrate that FedLEGO outperforms baselines on three datasets under both IID and Non-IID settings. Additionally, FedLEGO effectively reduces the adverse impact of using different public data and dynamically generates diverse personalized models in an automated manner.