Towards Efficient World Models

Eloi Alonso, Vincent Micheli, François Fleuret

Keywords: Deep learning, Reinforcement learning, World Models, Transformers

Abstract: Scaling up deep Reinforcement Learning (RL) agents beyond traditional benchmarks, without abundant computational resources, presents a significant challenge. Following recent developments in generative modelling, model-based RL positions itself as a strong contender to bring autonomous agents to new heights. In fact, the recently introduced IRIS agent provides evidence that advances in sequence modelling can be leveraged to build powerful world models. In the present work, we propose delta-IRIS, a new agent with a world model architecture that is amenable to scaling up to visually complex environments with longer time horizons. In the Crafter benchmark, delta-IRIS solves 16 out of 22 tasks after 10M frames of training, matching the current best method, DreamerV3.

Submission Number: 45