A Dynamical System Perspective for Escaping Sharp Local Minima in Equality Constrained Optimization Problems

Han Feng, Haixiang Zhang, Javad Lavaei

Abstract: This paper provides a dynamical system perspective on the escape of sharp local minima in constrained optimization problems. The dynamical system view models a perturbed projected first-order optimization algorithm and translates the problem of escaping local minima in constrained optimization problems to that of escaping regions of attraction of the corresponding dynamical system. We develop the notion of biased perturbation and show that it gives a quantitative view of the notion of small regions of attraction that can be escaped. As a counterpart, we explain why the dynamics is stable in a wide region of attraction around a strongly stable equilibrium. Numerical examples are provided to illustrate the usefulness of the developed concepts.