No fear of domain discrepancy

Siyuan Wu, Yipeng Zhou, Miao Hu, Di Wu

Abstract: One-shot Federated Learning (FL) has attracted tremendous attention for training a global model across decentralized devices in a single communication round. Despite the low communication cost, the efficiency of one-shot FL methods is often compromised by class imbalance conditions among clients, where the client models trained on skewed local distribution can mislead the global model. Meanwhile, auxiliary public data on the server is commonly used to distill client predictions. However, domain discrepancies between server and client data can hinder federated distillation, leading to suboptimal results. To address these issues, we propose OFedCD: One-Shot Federated Learning via Class-Aware Distillation. OFedCD treats each local data as a distinct domain and introduces a novel weighting strategy based on domain distances to aggregate reliable predictions. To enhance knowledge transfer efficiency, OFedCD employs bi-directional ensemble distillation, which aligns the global model with client prediction structures, including both inter-class and intra-class levels. We demonstrate the superiority of our method through domain adaptation generalization bound analysis and extensive experiments on benchmark datasets.

External IDs:doi:10.1007/978-981-96-2864-3_16