Stability and Generalization of Split Learning : Sequential or Federated

Yifei Liang, Yan Sun, Li Shen, Xiaochun Cao

Keywords: Spilt learning, Stability, Generalization

Abstract: Split Learning (SL) has emerged as a practical paradigm for training large models under privacy and systems constraints, showing strong performance on heterogeneous data and aligning well with LLM-era workloads. However, while convergence analyses for SL algorithms such as Sequential Split Learning (SSL) and Split Federated Learning (SFL) are well-established, their generalization bounds, especially those dependent on iteration-specific factors, remain largely unexplored, hindered by challenges like client drift and biased gradient estimates. In this work, we introduce the first theoretical framework for analyzing the generalization error of SL algorithms, leveraging an on-average stability approach to account for both local update drift and aggregation-induced errors. Our framework provides a novel connection between optimization and generalization, revealing how SSL and SFL differ in their stability profiles and generalization behavior. Specifically, we demonstrate that SSL excels in sparse client participation and long-horizon training, while SFL benefits from balanced participation in non-convex regimes, offering a clear guide for selecting the appropriate aggregation strategy. By deriving precise stability bounds for both convex and non-convex settings, we provide deep insights into the role of data heterogeneity, client drift, and aggregation mechanisms in SL. Extensive experiments on MNIST and CIFAR-10 benchmarks validate our theoretical predictions, highlighting the robustness and applicability of our framework across a range of practical scenarios.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 23163