Advancing Trajectory Optimization with Approximate Inference: Exploration, Covariance Control and Adaptive Risk

Joe Watson, Jan Peters

Abstract: Discrete-time stochastic optimal control remains a challenging problem for general, nonlinear systems under significant uncertainty, with practical solvers typically relying on the certainty equivalence assumption, replanning and/or extensive regularization. Control-as-inference is an approach that frames stochastic control as an equivalent inference problem, and has demonstrated desirable qualities over existing methods, namely in exploration and regularization. We look specifically at the input inference for control (I2C) algorithm, and derive three key characteristics that enable advanced trajectory optimization: An `expert' linear Gaussian controller that combines the benefits of open-loop optima and closed-loop variance reduction when optimizing for nonlinear systems, adaptive risk sensitivity for regularized exploration, and performing covariance control through specifying the terminal state distribution.