Learning Robust Representations for World Models without Reward Signals

Zeqiang Zhang, Fabian Wurzberger, Sebastian Gottwald, Daniel Alexander Braun

Keywords: Model-based Reinforcement Learning, Unsupervised Learning, World Models, Representation Learning

TL;DR: The paper proposes a hybrid unsupervised learning method for model-based RL that combines reconstruction, inverse dynamics, and contrastive learning to better capture task-relevant features in reward-free settings.

Abstract: Learning accurate and generalizable world models is a central challenge in model-based reinforcement learning (MBRL), particularly in reward-free settings where no task-specific supervision is available. In this paper, we investigate how different unsupervised objectives, including reconstruction, inverse dynamics, and contrastive learning, capture distinct components of the observation space, such as noise, background, controllable dynamics, and slow-changing factors. Building on this understanding, we introduce a hybrid representation learning approach that integrates the strengths of multiple objectives to better capture predictable and task-relevant structure. We design a controlled shape-based environment with disentangled latent factors to evaluate the robustness and utility of learned representations. Empirical results show that our method yields more informative and generalizable representations.

Confirmation: I understand that authors of each paper submitted to EWRL may be asked to review 2-3 other submissions to EWRL.

Serve As Reviewer: ~Zeqiang_Zhang1

Track: Regular Track: unpublished work

Submission Number: 149