Faster Accelerated First-order Methods for Convex Optimization with Strongly Convex Function Constraints

Zhenwei Lin, Qi Deng

Keywords: Convex optimization; Accelerated primal dual algorithm; Sparse Optimization

TL;DR: We propose faster accelerated primal-dual algorithms for minimizing a convex function subject to strongly convex function constraints.

Abstract:

In this paper, we introduce faster accelerated primal-dual algorithms for minimizing a convex function subject to strongly convex function constraints.
Prior to our work, the best complexity bound was $\mathcal{O}(1/{\varepsilon})$, regardless of the strong convexity of the constraint function.
It is unclear whether the strong convexity assumption can enable even better convergence results.
To address this issue, we have developed novel techniques to progressively estimate the strong convexity of the Lagrangian function.
Our approach, for the first time, effectively leverages the constraint strong convexity, obtaining an improved complexity of $\mathcal{O}(1/\sqrt{\varepsilon})$. This rate matches the complexity lower bound for strongly-convex-concave saddle point optimization and is therefore order-optimal.
We show the superior performance of our methods in sparsity-inducing constrained optimization, notably Google's personalized PageRank problem. Furthermore, we show that a restarted version of the proposed methods can effectively identify the optimal solution's sparsity pattern within a finite number of steps, a result that appears to have independent significance.

Primary Area: Optimization (convex and non-convex, discrete, stochastic, robust)

Submission Number: 13386