Analysis of the Whiplash gradient descent dynamics

Subhransu S. Bhattacharjee, Ian R. Petersen

Abstract: In this paper, we propose the Whiplash inertial gradient dynamics, a closed-loop optimization method that utilizes gradient information. We introduce the symplectic asymptotic convergence analysis for the Whiplash system for convex functions. We also introduce relaxation sequences to explain the nonclassical nature of the algorithm and an exploring heuristic variant of the Whiplash algorithm to escape saddle points, deterministically. We provide a practical methodology for analyzing convergence rates using integral constraint bounds and a novel Lyapunov rate method. Our results demonstrate polynomial and exponential rates of convergence for quadratic costs.