Bellman-Local Lyapunov Barriers for Exact Stationary Nash Learning in Discounted Perfect-Information Stochastic Games

Manoj Saravanan

Keywords: stochastic games, discounted perfect-information stochastic games, stationary Nash equilibrium, game-theoretic learning, Bellman-local dynamics, Lyapunov methods, local learning dynamics, computational complexity, PPAD, PLS, CLS

Abstract: We study whether exact stationary Nash equilibria in finite rational two-player discounted perfect-information stochastic games can be reached by universally defined local learning dynamics. We introduce a Bellman-local model in which an update rule may inspect the exact Bellman jet of the current stationary profile---the current policy, exact value vectors, exact one-step continuation values, exact deviation gains, and exact discounted occupancies---and is certified by a Bellman-local Lyapunov witness. Our main result shows that if there exists a universal Bellman-local strict-descent pair whose fixed points are exactly the stationary Nash equilibria on bounded-bit policy grids, then the exact stationary-Nash search problem belongs to $\mathrm{PLS}$; since this problem is already $\mathrm{PPAD}$-complete, this implies the complexity collapse $\mathrm{PPAD}=\mathrm{CLS}$. We further prove a quantitative strengthening: if the same Bellman-local descent admits polynomially bounded witness range and inverse-polynomial one-step progress, then simple iteration computes exact or constant-accuracy stationary Nash in deterministic polynomial time, implying $\mathrm{PPAD}\subseteq \mathrm{FP}$ at sufficiently small constant accuracy. These results identify a sharp frontier for game-theoretic learning in stochastic environments: Bellman locality and exact critic information alone do not suffice for a universal exact descent theory without additional structural assumptions.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 1