Automatic Tuning of Federated Learning Hyper-Parameters from System Perspective

Huanle Zhang, Mi Zhang, Xin Liu, Prasant Mohapatra, Michael Delucia

Keywords: federated learning, training preference, system perspective, hyper-parameter tuning

Abstract:

Federated Learning (FL) is a distributed model training paradigm that preserves clients' data privacy.
FL hyper-parameters significantly affect the training overheads in terms of time, computation, and communication.
However, the current practice of manually selecting FL hyper-parameters puts a high burden on FL practitioners since various applications prefer different training preferences. In this paper, we propose FedTuning, an automatic FL hyper-parameter tuning algorithm tailored to applications' diverse system requirements of FL training. FedTuning is lightweight and flexible, achieving an average of 41% improvement for different training preferences on time, computation, and communication compared to fixed FL hyper-parameters.

One-sentence Summary: An automatic tuning algorithm for adjusting federated learning hyper-parameters according to training preferences

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2110.03061/code)