Efficient GPU-Accelerated Global Optimization for Inverse Problems

Utkarsh, Vaibhav Kumar Dixit, Julian Samaroo, Avik Pal, Alan Edelman, Christopher Vincent Rackauckas

Keywords: Scientific Machine Learning, Inverse Problems, Global Optimization, GPU Computing

Abstract: This paper introduces a novel hybrid multi-start optimization strategy for solving inverse problems involving nonlinear dynamical systems and machine learning architectures, accelerated by GPU computing on both NVIDIA and AMD GPUs. The method combines Particle Swarm Optimization (PSO) and the Limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithms to address the challenges in parameter estimation for nonlinear dynamical systems. This hybrid strategy aims to leverage the global search capability of PSO and the efficient local convergence of L-BFGS. We experimentally show faster convergence by a factor of up to $8-30\times$ in a few non-convex problems with loss landscapes characterized by multiple local minima, which can cause regular optimization approaches to fail.

Submission Number: 89