Learning Dynamic State Abstractions for Model-Based Reinforcement LearningDownload PDF

Feb 15, 2018 (edited Feb 15, 2018)ICLR 2018 Conference Blind SubmissionReaders: Everyone
  • Abstract: A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed models that learn predictive and compact state representations, also called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment (ALE) from raw pixels. Furthermore, RL agents that use Monte-Carlo rollouts of these models as features for decision making outperform strong model-free baselines on the game MS_PACMAN, demonstrating the benefits of planning using learned dynamic state abstractions.
  • Keywords: generative models, probabilistic modelling, reinforcement learning, state-space models, planning
11 Replies

Loading