Optimal Noise Control of Sampling-Based Predictive Control for Inference Time Scaling

Benjamin Riviere, Joel W. Burdick

Keywords: Sampling-based planning, Optimal control, Inference-time computation

TL;DR: This paper proposes a novel search-based noise control for sampling-based predictive control.

Abstract:

Robots that are controlled by evaluating an offline trained model can fail in the presence of out-of-distribution data. Real-time optimization, also known as inference time scaling, can be a complementary approach to make learning-based systems robust to this error. One such method, sampling-based predictive control (SBPC), uses finite samples to approximate gradient steps in trajectory optimization. SBPC methods are gaining attention because they are less sensitive to dynamical modeling errors, allow for naturally parallelizable implementations, and they can be used in combination with pretrained models for inference time scaling and out-of-domain generalization. However, the performance of SBPC is dependent on hyperparameters, the most important of which is the noise model that controls the finite sample generation process. Whereas existing work considers the noise model as a problem-specific hyperparameter, we take the perspective of noise as a control input to the SBPC planning process, develop the corresponding optimal control problem, and propose a hierarchical search-and-sample solution. In particular, we parameterize the noise model with parameters that control the overall noise level and the noise level as a function of simulation time, and optimize these parameters over the solver iterations with Monte Carlo Tree Search.
Our approach removes hyperparameters, maintains finite-time theoretical convergence guarantees, and improves empirical performance on robot planning problems.

Submission Number: 41