Reinforced Imitation Learning from ObservationsDownload PDF

27 Sep 2018 (modified: 18 Oct 2019)ICLR 2019 Conference Blind SubmissionReaders: Everyone
  • Abstract: Imitation learning is an effective alternative approach to learn a policy when the reward function is sparse. In this paper, we consider a challenging setting where an agent has access to a sparse reward function and state-only expert observations. We propose a method which gradually balances between the imitation learning cost and the reinforcement learning objective. Built upon an existing imitation learning method, our approach works with state-only observations. We show, through navigation scenarios, that (i) an agent is able to efficiently leverage sparse rewards to outperform standard state-only imitation learning, (ii) it can learn a policy even when learner's actions are different from the expert, and (iii) the performance of the agent is not bounded by that of the expert due to the optimized usage of sparse rewards.
  • Keywords: imitation learning, state-only observations, self-exploration
10 Replies