Almost Sure Convergence of Nonlinear Stochastic Approximation Under General Moment Conditions

Anh Duc Nguyen, Quang Nguyen, Hoang H Nguyen, Siva Theja Maguluri

Keywords: Nonlinear Stochastic Approximation, Almost Sure Convergence, Truncation Method

TL;DR: We show almost sure convergence of stochastic approximation under general moment conditions using state-dependent truncation and Lyapunov drift method.

Abstract: We study the almost sure convergence of the Stochastic Approximation algorithm with diminishing step sizes $\alpha_n = \mathcal{O}\left(n^{-\xi}\right)$ for some $\xi \in (0,1]$ under a general noise moment assumption and a contractive operator. In particular, we show that for a martingale difference noise with $p$-th order integrability, we have almost sure convergence whenever $\max ( 1/2 , 1/p) < \xi < 1$. Our result generalizes (weighted) Law of Large Numbers and the almost sure convergence results in \cite{neurodynamic, Borkar2008StochasticAA, zaiwei-envelope}. To establish such results, we introduce a state-dependent moving truncation coupled with a fine-grained Lyapunov drift analysis. This approach effectively manages the bias from truncated terms and addresses the challenges posed by multiplicative noise, allowing us to relax the stringent assumptions often found in the literature.

Submission Number: 56