Learning Dynamic State Abstractions for Model-Based Reinforcement LearningDownload PDF

15 Feb 2018, 21:29 (edited 10 Feb 2022)ICLR 2018 Conference Blind SubmissionReaders: Everyone
  • Keywords: generative models, probabilistic modelling, reinforcement learning, state-space models, planning
  • Abstract: A key challenge in model-based reinforcement learning (RL) is to synthesize computationally efficient and accurate environment models. We show that carefully designed models that learn predictive and compact state representations, also called state-space models, substantially reduce the computational costs for predicting outcomes of sequences of actions. Extensive experiments establish that state-space models accurately capture the dynamics of Atari games from the Arcade Learning Environment (ALE) from raw pixels. Furthermore, RL agents that use Monte-Carlo rollouts of these models as features for decision making outperform strong model-free baselines on the game MS_PACMAN, demonstrating the benefits of planning using learned dynamic state abstractions.
  • Data: [Arcade Learning Environment](https://paperswithcode.com/dataset/arcade-learning-environment)
11 Replies