HeteroSFL: Split Federated Learning With Heterogeneous Clients and Non-IID Data

Xing Chen, Jingtao Li, Deliang Fan, Chaitali Chakrabarti

Abstract: Split Federated Learning (SFL) is an emerging privacy-preserving decentralized learning scheme which splits a machine learning model between client and server such that most of the computations are offloaded to the server. While SFL has low computation cost on the client side, it has high communication cost. Existing SFL schemes focus on reducing the communication cost for homogeneous clients. However, a more realistic scenario is when clients are heterogeneous and process data with different distributions. In this article, we focus on client-level heterogeneity caused by different communication data rates. We propose HeteroSFL, the first SFL framework with heterogeneous clients that handles non-IID data with label distribution skew across groups of clients. HeteroSFL compresses data with different compression factors in low-end and high-end groups using narrow and wide bottleneck layers (BL), respectively. It provides a mechanism to address the challenge of aggregating different-sized BL models and utilizes bidirectional knowledge sharing (BDKS) to address the overfitting issue caused by different label distributions across high- and low-end groups of clients. Our experimental results show that HeteroSFL achieves significant training time reduction with minimum accuracy loss compared to competing methods. Specifically, it can reduce the training time of SFL by $16\times $ to $256\times $ with 1.24% to 5.59% accuracy loss for VGG11 on CIFAR10 for non-IID data.

External IDs:dblp:journals/iotj/ChenLFC25