Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules

John Vastola, Samuel J. Gershman, Kanaka Rajan

Keywords: gradient descent, optimization, momentum, natural gradient, Adam, learning rules, theory, optimal control, normative framework, variational principles, continual learning

TL;DR: We derive well-known learning rules from an objective that casts learning rules as policies for navigating uncertain loss landscapes

Abstract: Learning rules—prescriptions for updating model parameters to improve performance—are typically assumed rather than derived. Why do some learning rules work better than others, and under what assumptions can a given rule be considered optimal? We propose a theoretical framework that casts learning rules as policies for navigating (partially observable) loss landscapes, and identifies optimal rules as solutions to an associated optimal control problem. A range of well-known rules emerge naturally within this framework under different assumptions: gradient descent from short-horizon optimization, momentum from longer-horizon planning, natural gradients from accounting for parameter space geometry, non-gradient rules from partial controllability, and adaptive optimizers like Adam from online Bayesian inference of loss landscape shape. We further show that continual learning strategies like weight resetting can be understood as optimal responses to task uncertainty. By unifying these phenomena under a single objective, our framework clarifies the computational structure of learning and offers a principled foundation for designing adaptive algorithms.

Primary Area: Theory (e.g., control theory, learning theory, algorithmic game theory)

Submission Number: 23109