Privacy Distributed Constrained Optimization Over Time-Varying Unbalanced Networks and Its Application in Federated Learning

Mengli Wei, Wenwu Yu, Duxin Chen, Mingyu Kang, Guang Cheng

Abstract: This paper investigates a class of constrained distributed zeroth-order optimization (ZOO) problems over time-varying unbalanced graphs while ensuring privacy preservation among individual agents. Not taking into account recent progress and addressing these concerns separately, there remains a lack of solutions offering theoretical guarantees for both privacy protection and constrained ZOO over time-varying unbalanced graphs. We hereby propose a novel algorithm, termed the differential privacy (DP) distributed push-sum based zeroth-order constrained optimization algorithm (DP-ZOCOA). Operating over time-varying unbalanced graphs, DP-ZOCOA obviates the need for supplemental suboptimization problem computations, thereby reducing overhead in comparison to distributed primary-dual methods. DP-ZOCOA is specifically tailored to tackle constrained ZOO problems over time-varying unbalanced graphs, offering a guarantee of convergence to the optimal solution while robustly preserving privacy. Moreover, we provide rigorous proofs of convergence and privacy for DP-ZOCOA, underscoring its efficacy in attaining optimal convergence without constraints. To enhance its applicability, we incorporate DP-ZOCOA into the federated learning framework and formulate a decentralized zeroth-order constrained federated learning algorithm (ZOCOA-FL) to address challenges stemming from the time-varying imbalance of communication topology. Finally, the performance and effectiveness of the proposed algorithms are thoroughly evaluated through simulations on distributed least squares (DLS) and decentralized federated learning (DFL) tasks.

External IDs:dblp:journals/ieeejas/WeiYCKC25