Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency PlanningDownload PDF

04 Mar 2022, 07:18 (modified: 14 Apr 2022, 15:50)ICLR 2022 GPL PosterReaders: Everyone
Keywords: Reinforcement Learning, Contingency Planning, Hierarchical Control
TL;DR: We present a method to learn contingency plans, and a controller that switches between optimal and contingency strategies to find a sweet spot between safety and performance in autonomous driving applications.
Abstract: When learning to act in a stochastic, partially observable environment, an intelligent agent should be prepared to anticipate a change in its belief of the environment state, and be capable of adapting its actions on-the-fly to changing conditions. As humans, we are able to form contingency plans when learning a task with the explicit aim of being able to correct errors in the initial control, and hence prove useful if ever there is a sudden change in our perception of the environment which requires immediate corrective action. This is especially the case for autonomous vehicles (AVs) navigating real-world situations where safety is paramount, and a strong ability to react to a changing belief about the environment is truly needed. In this paper we explore an end-to-end approach, from training to execution, for learning robust contingency plans and combining them with a hierarchical planner to obtain a robust agent policy in an autonomous navigation task where other vehicles’ behaviours are unknown, and the agent’s belief about these behaviours is subject to sudden, last-second change. We show that our approach results in robust, safe behaviour in a partially observable, stochastic environment, generalizing well over environment dynamics not seen during training.
1 Reply