Switching Gradient Methods for Constrained Federated Optimization

Antesh Upadhyay, Sang Bin Moon, Abolfazl Hashemi

Keywords: Federated Learning, Distributed Optimization, Constrained Optimization

TL;DR: We provide algorithms to solve constrained federated learning problems via switching gradient methods under compression.

Abstract: Constrained optimization problems arise in federated learning (FL) settings, where a global objective must be minimized subject to a functional constraint aggregated across clients. We introduce Federated Switching Gradient Methods (FedSGM), a primal-only, projection-free algorithm for federated constrained optimization. By extending switching gradient methods to the federated setting, FedSGM avoids the inner solves and penalty tuning required by dual or penalty-based methods, enabling lightweight and scalable deployment. Our analysis addresses three practical challenges simultaneously: (i) multi-step local updates to accommodate heterogeneous client compute, (ii) unbiased uplink compression to mitigate communication costs, and (iii) both hard and soft switching between objective and constraint gradients. We provide the first convergence guarantees for constrained FL that hold under these combined settings, recovering known centralized rates in special cases. In particular, we show that soft switching, recently proposed in the centralized literature, retains convergence guarantees while offering improved empirical stability near the constraint boundary.

Submission Number: 86