From Observations to Events: Event-Aware World Models for Reinforcement Learning

Zhao-Han Peng, Shaohui Li, Zhi Li, Shulan Ruan, Yu LIU, You He

Keywords: model-based reinforcement learning, online learning, reinforcement learning

TL;DR: EAWM is an event-aware world model that automatically generates and segments events from raw observations to make model-based RL more robust and generalizable, achieving 10–45% gains and setting new state-of-the-art results across diverse benchmarks.

Abstract: While model-based reinforcement learning (MBRL) improves sample efficiency by learning world models from raw observations, existing methods struggle to generalize across structurally similar scenes and remain vulnerable to spurious variations such as textures or color shifts. From a cognitive science perspective, humans segment continuous sensory streams into discrete events and rely on these key events for decision-making. Motivated by this principle, we propose the Event-Aware World Model (EAWM), a general framework that learns event-aware representations to streamline policy learning without requiring handcrafted labels. EAWM employs an automated event generator to derive events from raw observations and introduces a Generic Event Segmentor (GES) to identify event boundaries, which mark the start and end time of event segments. Through event prediction, the representation space is shaped to capture meaningful spatio-temporal transitions. Beyond this, we present a unified formulation of seemingly distinct world model architectures and show the broad applicability of our methods. Experiments on Atari 100K, Craftax 1M, and DeepMind Control 500K, DMC-GB2 500K demonstrate that EAWM consistently boosts the performance of strong MBRL baselines by 10\%–45\%, setting new state-of-the-art results across benchmarks. Our code is released at [https://github.com/MarquisDarwin/EAWM](https://github.com/MarquisDarwin/EAWM).

Primary Area: reinforcement learning

Submission Number: 11785