Bayesian Reinforcement Learning for Single-Episode Missions in Partially Unknown EnvironmentsDownload PDF

16 Jun 2022, 10:45 (modified: 03 Oct 2022, 17:14)CoRL 2022 PosterReaders: Everyone
Student First Author: yes
Keywords: Planning under Uncertainty, Gaussian Processes, Single-Episode Bayesian Reinforcement Learning
TL;DR: We propose a unified Bayes-optimal framework for single-mission robot planning in unknown environments, and evaluate on two realistic simulated environments.
Abstract: We consider planning for mobile robots conducting missions in real-world domains where a priori unknown dynamics affect the robot’s costs and transitions. We study single-episode missions where it is crucial that the robot appropriately trades off exploration and exploitation, such that the learning of the environment dynamics is just enough to effectively complete the mission. Thus, we propose modelling unknown dynamics using Gaussian processes, which provide a principled Bayesian framework for incorporating online observations made by the robot, and using them to predict the dynamics in unexplored areas. We then formulate the problem of mission planning in Markov decision processes under Gaussian process predictions as Bayesian model-based reinforcement learning. This allows us to employ solution techniques that plan more efficiently than previous Gaussian process planning methods are able to. We empirically evaluate the benefits of our formulation in an underwater autonomous vehicle navigation task and robot mission planning in a realistic simulation of a nuclear environment.
Supplementary Material: zip
13 Replies