World Model Anomaly Detection with a Latent Linear Prior

TMLR Paper7619 Authors

Abstract: Model-based reinforcement learning (MBRL) learns world models—internal simulators of environment dynamics—to plan by imagining future trajectories. However, when these models incorrectly predict state transitions, they generate unrealistic states that mislead agents into learning delusional policies. Inspired by human vision, we propose anomaly detection in world model with \textbf{L}inear \textbf{P}rior (LP), a three‐stage approach that 1) enforces a lightweight linear prior on successive latent states, 2) flags generated states that deviate from this prior, and 3) removes their contribution during agent learning. On the challenging Atari100k benchmark, LP-assisted GRU and Transformer based MBRL agents achieve competitive results while requiring less value updates with minimal additional computational cost. Notably, by suppressing false value updates with LP, DreamerV3 boosts human-normalized mean score by 9% while requiring less than 90% of the value updates. We release our implementation at https://anonymous.4open.science/r/lp-dreamer.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Li_Erran_Li1

Submission Number: 7619