Revisiting the Geometrically Decaying Step Size: Linear Convergence for Smooth or Non-Smooth Functions

Jihun Kim

Keywords: Geometrically Decaying Step Size, Linear Convergence, Non-Smooth Analysis

TL;DR: We revisit the geometrically decaying step size given a positive condition number, under which a locally Lipschitz function shows linear convergence.

Abstract: We revisit the geometrically decaying step size given a positive inverse condition number, under which a locally Lipschitz function shows linear convergence. The positivity does not require the function to satisfy convexity, weak convexity, quasar convexity, or sharpness, but instead amounts to a property strictly weaker than the assumptions used in existing works ($e.g.$, weak convexity + sharpness). We propose a clean and simple subgradient descent algorithm that requires minimal knowledge of problem constants, applicable to either smooth or non-smooth functions.

Submission Number: 22