FedPLT: Scalable, Resource-Efficient, and Heterogeneity-Aware Federated Learning via Partial Layer Training

Ahmad Dabaja, Rachid El-Azouzi

Abstract:

Federated Learning (FL) has gained significant attention in distributed machine learning by enabling collaborative
model training across decentralized devices while preserving data privacy. Although extensive research has addressed statistical
data heterogeneity, FL still faces several challenges, including high communication and computation overheads, energy inefficiency,
and severe device heterogeneity, which require further investigation. Prior work has addressed these issues through submodel
training and partial parameter updates. However, such methods often suffer from inconsistent parameter distributions across clients, inaccurate global loss estimation, and increased bias and variance. In this paper, we propose FedPLT (Federated Learning with Partial Layer Training), a novel and structured partial training approach that divides neural network layers into equal-sized sub-layers and assigns them to clients based on their communication and computational capacities using a fixed assignment strategy. This ensures balanced parameter distribution, reduces variance, and supports varying model sizes across devices, making FedPLT well-suited for resource-constrained environments. In addition, we examine the performance of FedPLT when combined with optimal client sampling and show that this integration enhances federated learning performance by reducing sampling variance under the same communication constraints. Through extensive experiments, we show that FedPLT achieves performance comparable to, or even superior to, that of fullmodel training (i.e., FedAvg), while requiring significantly fewer trainable parameters per client. It also outperforms existing methods in the literature under highly heterogeneous systems by efficiently adapting to clients’ resource constraints and reducing the number of stragglers. Moreover, FedPLT can reduce the trained parameters by up to 65–85%, leading to substantial savings in communication costs, while maintaining the same level of performance as full-model training.