OFASD: One-Shot Federated Anisotropic Scaling Distillation

Shunsei Koshibuchi, Hiroshi Kera, Kazuhiko Kawamoto

Keywords: one-shot federated learning, task arithmetic, knowledge distillation, data heterogeneity

Abstract: One‐shot federated learning often suffers from degraded performance under heterogeneous client data. To address this, we propose a task arithmetic-based knowledge integration method that applies anisotropic scaling. We optimize the scaling coefficients via knowledge distillation leveraging training data and ensemble outputs of local models. Using a ResNet-18 pre-trained on ImageNet-1K, we evaluate our method on CIFAR-10, CIFAR-100, and SVHN, each split according to a Dirichlet distribution. Our method outperforms the baseline across varying heterogeneity levels and achieves high accuracy with minimal training time.

Submission Number: 29