Convergence Analysis of Linear Coupling with Inexact Proximity Operator

Qiang Zhou, Sinno Pan

Keywords: linear coupling, proximal operator

TL;DR: This work presents a convergence analysis for linear coupling with inexact proximal operator.

Abstract: Linear coupling is recently proposed to accelerate first-order algorithms by linking gradient descent and mirror descent together, which is able to achieve an accelerated convergence rate. This work focuses on the convergence analysis of linear coupling for convex composite minimization when a proximal operator cannot be exactly computed. It is of particular interest to study the convergence of linear coupling because it not only achieves the accelerated convergence rate for first-order algorithms but also works for generic norms. We present convergence analysis of linear coupling by allowing the proximal operator to be computed up to a certain precision. Our analysis illustrates that the accelerated convergence rate of linear coupling with an inexact proximal operator can be preserved if the error sequence of the inexact proximal operator decreases in a sufficiently fast rate. More importantly, our analysis leads to better bounds than existing works with inexact proximal operators. Experiment results on several real-world datasets verify our theoretical results.