Context and Diversity Matter: The Emergence of In-Context Learning in World Models

Fan Wang, ZHIYUAN CHEN, YUXUAN ZHONG, Sunjian Zheng, Pengtao Shao, Bo Yu, Shaoshan Liu, Jianan Wang, Ning Ding, Yang Cao, Yu Kang

Keywords: In-Context Learning; World Models

TL;DR: We formalize, bound, and validate in-context environment learning, showing that long-context, diverse-input world models can self-adapt by recognizing or learning new dynamics without parameter updates.

Abstract: The capability of predicting environmental dynamics underpins both biological neural systems and general embodied AI in adapting to their surroundings. Yet prevailing approaches rest on static world models that falter when confronted with novel or rare configurations. We investigate in-context learning (ICL) of world models, shifting attention from zero-shot performance to the growth and asymptotic limits of the world model. Our contributions are three-fold: (1) we formalize ICL of a world model and identify two core mechanisms: environment recognition (ER) and environment learning (EL); (2) we derive error upper-bounds for both mechanisms that expose how the mechanisms emerge; and (3) we empirically confirm that distinct ICL mechanisms exist in the world model, and we further investigate how data distribution and model architecture affect ICL in a manner consistent with theory. These findings demonstrate the potential of self-adapting world models and highlight the key factors behind the emergence of EL/ER, most notably the necessity of long context and diverse environments. The codes are available at https://github.com/airs-cuhk/airsoul/tree/main/projects/MazeWorld.

Primary Area: transfer learning, meta learning, and lifelong learning

Submission Number: 11055