Hierarchical Reinforcement Learning and Model Predictive Control for Strategic Motion Planning in Autonomous Racing

Rudolf Reiter, Jasper Hoffmann, Joschka Boedecker, Moritz Diehl

Abstract:

We present an approach for safe trajectory planning, where a strategic task related to autonomous racing is learned sample efficiently within a simulation environment. A high-level policy, represented as a neural network, outputs a reward specification that is used within the objective of a parametric nonlinear model predictive controller. We can guarantee safe and feasible trajectories by including constraints and vehicle kinematics in the nonlinear program. Compared to classical reinforcement learning, our approach restricts exploration to safe trajectories, starts with good prior performance, and yields complete trajectories that can be passed to a tracking lowest-level controller.
We validate the performance of our algorithm in simulation and show how it learns to overtake and block other vehicles efficiently.

Format: Long format (up to 8 pages + refs, appendix)

Publication Status: Yes

Submission Number: 40