Effectively Heterogeneous Federated Learning: A Pairing and Split Learning Based Approach

Jinglong Shen, Xiucheng Wang, Nan Cheng, Longfei Ma, Conghao Zhou, Yuan Zhang

Abstract: Federated Learning (FL) is a promising paradigm widely used in privacy-preserving machine learning. It enables distributed devices to collaboratively train a model while avoiding data transfer between clients. Nevertheless, FL suffers from bottlenecks in training speed due to client heterogeneity, resulting in increased training latency and server aggregation lagging. To address this issue, a novel Split Federated Learning (SFL) framework is proposed. It pairs clients with different computational resources based on their computational resources and inter-client communication rates. The neural network model is split into two parts at the logical level, and each client computes only its assigned part using Split Learning (SL) to accomplish forward inference and backward training. Besides, a heuristic greedy algorithm is proposed to effectively deal with the client pairing problem by reconstructing the training latency optimization as a graph edge selection problem. Simulation results show that the proposed method can significantly improve the FL training speed and achieve high performance in both independent identical distribution (IID) and Non-IID data distribution.

External IDs:dblp:conf/globecom/ShenWCMZZ23