MARINA-P: Superior Performance in Nonsmooth Federated Optimization with Adaptive Stepsizes

ICLR 2026 Conference Submission25406 Authors

Keywords: Federated Learning, Communication-efficient non-smooth optimization, Adaptive Stepsizes

TL;DR: We extend MARINA-P and EF21-P to non-smooth distributed optimization, introduce adaptive stepsizes, and show MARINA-P with permutation compressors outperforms EF21-P in non-smooth settings

Abstract: Non-smooth communication-efficient federated optimization remains largely unexplored theoretically, despite its importance in machine learning applications. We consider a setup focusing on optimizing downlink communication by improving state-of-the-art schemes like EF21-P [Gruntkowska et al., 2023] and MARINA-P [Gruntkowska et al., 2024] in the non-smooth convex setting. Our key contributions include extending the non-smooth convex theory of EF21-P from single-node to distributed settings and generalizing MARINA-P to non-smooth convex optimization. For both algorithms, we prove optimal $\mathcal{O}(1/\sqrt{T})$ convergence rates under standard assumptions and establish matching communication complexity bounds with classical subgradient methods. We provide theoretical guarantees under constant, decreasing, and adaptive (Polyak-type) stepsizes. Our experiments demonstrate MARINA-P’s superior performance with correlated compressors in both smooth non-convex and non-smooth convex settings. This work presents the first theoretical analysis of distributed non-smooth optimization with server-to-worker compression, including a comprehensive analysis for various stepsize schemes.

Primary Area: optimization

Submission Number: 25406