Monte-Carlo Tree Search vs. Model-Predictive Controller: A Track-Following Example

Hengshuai Yao, Masoud S. Nosrati, Kasra Rezaee

Oct 11, 2017 (modified: Oct 12, 2017) NIPS 2017 Workshop MLITS Submission readers: everyone
  • Abstract: Monte-Carlo Tree Search (MCTS) has achieved remarkable success in the game of Go. However, most success of MCTS is in games where actions are discrete. For automous driving, the vehicle action such as throttle and steering angle is continuous. To fill the gap, we propose an MCTS algorithm for continuous actions, and used it specially for a track-following scenerio. We compared MCTS with a standard Model Predictive Controller (MPC) on the Udacity simulator. Using the same cost function and system model, this MCTS algorithm achieves a much lower cost than MPC. MCTS drives with an adaptive speed, as well as exhibits a braking behavior in sharp turns. MPC drives a nearly constant speed regardless of the curvy track.
  • TL;DR: We propose an MCTS algorithm for continuous actions, and used it specially for a track-following scenerio in autonomous driving.
  • Keywords: Reinforcement Learning, Autonomous Driving

Loading