Information Geometry of Exponentiated Gradient: Convergence Beyond L-Smoothness

Yara Elshiaty, Ferdinand Vanmaele, Stefania Petra

Abstract: We study the minimization of smooth, possibly nonconvex functions over the positive orthant, a key setting in Poisson inverse problems, using the (unnormalized) exponentiated gradient (EG) method. Interpreting EG as Riemannian gradient descent (RGD) with the e-Exp map from information geometry as a retraction, we prove global convergence of the function values under weak assumptions – without the need for L-smoothness – and finite termination of Riemannian Armijo line search. Numerical experiments, including an accelerated variant, highlight EG’s practical advantages, such as faster convergence compared to RGD based on interior-point geometry.

External IDs:doi:10.1007/978-3-031-92369-2_4