Unified linear convergence of first-order primal-dual algorithms for saddle point problems

Fan Jiang, Zhongming Wu, Xingju Cai, Hongchao Zhang

Abstract: In this paper, we study the linear convergence of several well-known first-order primal-dual methods for solving a class of convex-concave saddle point problems. We first unify the convergence analysis of these methods and prove the O(1/N) convergence rates of the primal-dual gap generated by these methods in the ergodic sense, where N counts the number of iterations. Under a mild calmness condition, we further establish the global Q-linear convergence rate of the distances between the iterates generated by these methods and the solution set, and show the R-linear rate of the iterates in the nonergodic sense. Moreover, we demonstrate that the matrix games, fused lasso and constrained TV- $$\ell _2$$ ℓ 2 image restoration models as application examples satisfy this calmness condition. Numerical experiments on fused lasso demonstrate the linear rates for these methods.