Topological Control of Optimization Dynamics on Evolving Manifolds

Xingrui Gu

Keywords: Information Geometry; Topology; Optimizer; Learning Dynamics

TL;DR: We propose TAGD, an optimizer-agnostic layer that estimates a low-rank update subspace from recent gradients and uses topological feedback to suppress unstable orthogonal drift.

Abstract: Machine learning optimization is usually framed as a sequence of local responses to the current gradient. In high-dimensional nonconvex training, however, it is more naturally understood as a noise-driven process of trajectory evolution. Motivated by symplectic reduction, which compresses high-dimensional dynamics onto a lower-dimensional effective manifold while preserving essential structure, we propose Trajectory-Adaptive Geometric Damping (TAGD), a geometry-topology control framework for first-order optimization. TAGD uses recent gradient history to estimate a low-rank effective subspace via online Gram-PCA, decomposes each update into tangential and normal components, and adaptively regulates normal suppression through multiscale topological stability feedback. This yields a trajectory-level control mechanism on a low-dimensional effective submanifold, rather than a purely local update rule. Theoretically, under local Fisher geometry, we show that the resulting update reduces information-geometric error, tightens the one-step descent bound, and remains robust to finite-window subspace estimation error; we further show that topological feedback acts as an online proxy for an otherwise unobservable orthogonal bias-variance tradeoff. Empirically, across CIFAR-10 and WikiText-103, TAGD improves performance in 11 of 12 settings without modifying the internal state of the base optimizer. On WikiText-103, relative to AdamW, it improves average best validation perplexity by 8.26\% on LLaMA-style models and 5.96\% on GPT-J-style models, for an overall gain of 7.11\%. Trajectory and local loss-landscape visualizations further show that TAGD enters low-loss channels earlier, reduces lateral drift, and in some cases alters basin selection. Overall, TAGD unifies information geometry, low-rank subspace control, and topological stability feedback within an online optimization framework, offering a trajectory-centric view of deep learning optimization.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 13