A Convex Approach to Stochastic Optimal Control Using Linear Operators

Umesh Vaidya, Bowen Huang

Abstract: The paper is about the optimal control of a stochastic dynamical system. We provide a convex formulation to the optimal control problem involving a stochastic dynamical system. The convex formulation is made possible by writing the stochastic optimal control problem in the dual space of densities involving the Fokker-Planck or Perron-Frobenius generator for a stochastic system. The convex formulation leads to an infinite-dimensional convex optimization problem for optimal control. We exploit Koopman and Perron-Frobenius generators' duality for the stochastic system to construct the finite-dimensional approximation of the infinite-dimensional convex problem. We present simulation results to demonstrate the application of the developed framework.