An optimized first-order method for image restoration

Donghwan Kim, Jeffrey A. Fessler

Abstract: First-order methods are used widely for large scale optimization problems in signal/image processing and machine learning, because their computation depends mildly on the problem dimension. Nesterov's fast gradient method (FGM) has the optimal convergence rate among first-order methods for smooth convex minimization; its extension to non-smooth case, the fast iterative shrinkage-thresholding algorithm (FISTA), also satisfies the optimal rate; thus both algorithms have gained great interest. We recently introduced a new optimized gradient method (OGM) (for smooth convex functions) having a theoretical convergence speed that is 2× faster than Nesterov's FGM. This paper further discusses the convergence analysis of OGM and explores its fast convergence on an image restoration problem using a smoothed total variation (TV) regularizer. In addition, we empirically investigate the extension of OGM to nonsmooth convex minimization for image restoration with l <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -sparsity regularization.