Proximal Newton-type methods for convex optimization

Jason D. Lee, Yuekai Sun, Michael A. Saunders

Abstract: We seek to solve convex optimization problems in composite form: minimizex∈ℝn f (x) := g(x) + h(x), where g is convex and continuously differentiable and h : ℝn → ℝ is a convex but not necessarily differentiable function whose proximal mapping can be evaluated efficiently. We derive a generalization of Newton-type methods to handle such convex but nonsmooth objective functions. We prove such methods are globally convergent and achieve superlinear rates of convergence in the vicinity of an optimal solution. We also demonstrate the performance of these methods using problems of relevance in machine learning and statistics.