Understanding Lookahead Dynamics Through Laplace Transforms

Aniket Sanyal, Tatjana Chavdarova

Keywords: Machine Learning, Optimization, Saddle Point Optimization, Games, Control Theory

Abstract: We introduce a frequency-domain framework for convergence analysis of hyper parameters in game optimization, leveraging High-Resolution Differential Equations (HRDEs) and Laplace transforms. Focusing on the Lookahead algorithm--parameterized by gradient steps $k$ and averaging coefficient $\alpha$--we reformulate oscillatory dynamics in bilinear games from the discrete time domain into the frequency domain, enabling precise convergence criteria. Our higher-precision $\mathcal{O}(\gamma^2)$-HRDE models derive tighter criteria, while even simplified $\mathcal{O}(\gamma)$-HRDE models retain practical utility: they prioritize actionable hyperparameter selection over analytically exhaustive expressions. Validated empirically in discrete-time dynamics, this approach potentially extends locally linear operators, bridging scalability and stability in game-theoretic training.

Student Paper: Yes

Submission Number: 95