Neural Policy Iteration for Stochastic Optimal Control: A Physics-Informed Approach

ICLR 2026 Conference Submission22269 Authors

Keywords: Hamilton--Jacobi--Bellman equation, stochastic optimal control, reinforecment learning, physic-informed neural network

Abstract: We propose a physics-informed neural network policy iteration (PINN-PI) framework for solving stochastic optimal control problems governed by second-order Hamilton--Jacobi--Bellman (HJB) equations. At each iteration, a neural network is trained to approximate the value function by minimizing the residual of a linear PDE induced by a fixed policy. This linear structure enables systematic $L^2$ error control at each policy evaluation step, and allows us to derive explicit Lipschitz-type bounds that quantify how value gradient errors propagate to the policy updates. This interpretability provides a theoretical basis for evaluating the quality of policy during training. Our method extends recent deterministic PINN-based approaches to stochastic settings, inheriting the global exponential convergence guarantees of classical policy iteration under mild conditions. We demonstrate the effectiveness of our method on several benchmark problems, including stochastic cartpole, pendulum problems and high-dimensional linear quadratic regulation (LQR) problems in up to 10D.

Primary Area: applications to physical sciences (physics, chemistry, biology, etc.)

Submission Number: 22269