The Hidden Cost of Approximation in Online Mirror Descent

Ofir Schlisselberg, Uri Sherman, Tomer Koren, Yishay Mansour

Keywords: Mirror Descent, Convex Optimization, Stochastic Optimization

TL;DR: We study inexact online mirror descent and characterize the relation between curvature properties of the regularizer and OMD's robustness to approximation errors.

Abstract:

Online mirror descent (OMD) is a fundamental algorithmic paradigm that underlies many algorithms in optimization, machine learning and sequential decision-making. The OMD iterates are defined as solutions to optimization subproblems which, oftentimes, can be solved only approximately, leading to an \emph{inexact} version of the algorithm. Nonetheless, existing OMD analyses typically assume an idealized error free setting, thereby limiting our understanding of performance guarantees that should be expected in practice. In this work we initiate a systematic study into inexact OMD, and uncover an intricate relation between regularizer smoothness and robustness to approximation errors. When the regularizer is uniformly smooth, we establish a tight bound on the excess regret due to errors.
Then, for barrier regularizers over the simplex and its subsets, we identify a sharp separation: negative entropy requires exponentially small errors to avoid linear regret, whereas log-barrier and Tsallis regularizers remain robust even when the errors are only polynomial. Finally, we show that when the losses are stochastic and the domain is the simplex, negative entropy regains robustness—but this property does not extend to all subsets, where exponentially small errors are again necessary to avoid suboptimal regret.

Submission Number: 58