A Bayesian Approach to Robust Inverse Reinforcement LearningDownload PDF

Published: 30 Aug 2023, Last Modified: 07 Oct 2023CoRL 2023 PosterReaders: Everyone
Keywords: Inverse Reinforcement Learning, Bayesian Inference, Robustness
Abstract: We consider a Bayesian approach to offline model-based inverse reinforcement learning (IRL). The proposed framework differs from existing offline model-based IRL approaches by performing simultaneous estimation of the expert's reward function and subjective model of environment dynamics. We make use of a class of prior distributions which parameterizes how accurate the expert’s model of the environment is to develop efficient algorithms to estimate the expert's reward and subjective dynamics in high-dimensional settings. Our analysis reveals a novel insight that the estimated policy exhibits robust performance when the expert is believed (a priori) to have a highly accurate model of the environment. We verify this observation in the MuJoCo environments and show that our algorithms outperform state-of-the-art offline IRL algorithms.
Student First Author: yes
Supplementary Material: zip
Instructions: I have read the instructions for authors (https://corl2023.org/instructions-for-authors/)
Code: https://github.com/rw422scarlet/bmirl_tf
Publication Agreement: pdf
Poster Spotlight Video: mp4
14 Replies