Diffusion World Models

Eloi Alonso, Adam Jelley, Anssi Kanervisto, Tim Pearce

Primary Area: reinforcement learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: World models, diffusion models, reinforcement learning, generative modeling

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Abstract: World models constitute a powerful and versatile tool for decision-making. Through their ability to predict future states of the world, they can replace environments for safe and fast simulation, and/or be leveraged for search at decision time. Advances in generative modeling have led to the development of new world models, that operate in visual environments with challenging dynamics. However, recurrent methods lack visual fidelity, and autoregressive approaches scale poorly with visual complexity. Inspired by the recent success of diffusion models for image generation, we introduce Diffusion World Models (DWM), a new approach to world modeling that offers a favorable trade-off between speed and quality. Through qualitative and quantitative experiments in a 3D videogame, real-world motorway driving, and RL environments, we show that Diffusion World Models are an excellent choice for simulating visually complex worlds.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 2509