FedProx-based heterogeneity-aware parameter-free federated learning

Kenta Niwa, Tomoya Murata, Takumi Fukami

Keywords: Federated learning, parameter-free optimization, FedProx

TL;DR: FedProx-based heterogeneity-aware parameter-free federated learning algorithms are proposed.

Abstract: We propose parameter-free Federated Learning (FL) algorithms based on FedProx. Learning rate-free optimization has been studied in single-node settings, with DoG and its extension DoWG exhibiting strong theoretical and empirical performance. To exploit their success in multi-node FL, we leverage a key insight: a structural similarity between the lemmas for convergence analyses of DoG/DoWG and those of the proximal point algorithm that underlies FedProx. Based on this, we propose two novel FedProx-based algorithms--FedProxLoD and FedProxWLoD--which adaptively determine the proximal weight, serving as FL analogues of DoG and DoWG. We show tight heterogeneity-aware convergence rates for parameter-free FL that explicitly reflect the impact of data heterogeneity across clients and demonstrate that the proposed algorithms can outperform DoG and DoWG as heterogeneity decreases. Through large-scale numerical experiments on both convex and non-convex models, we validate the effectiveness of the proposed methods. Notably, FedProxWLoD achieved competitive performance with pre-tuned beseline algorithms under moderate data heterogeneity settings.

Primary Area: optimization

Submission Number: 12414