Position: Hierarchical World Models with Causal Curation for Generalizing Agents

Fei Dai, Hanqi Zhou, Alison Gopnik

Keywords: empowerment, causal learning, HRL, generalization, formal and informal reasoning of LLM agent

Abstract:

Current reinforcement learning agents excel at solving narrow tasks yet struggle to generalize beyond the environments they were trained in, which is often attributed to a lack of causal understanding. While learning world models can provide foresight, they are often brittle, whereas pure causal discovery remains intractable in complex settings. In this position paper, we argue that the path to a general-purpose agent lies not in building a single, monolithic world model, but in actively curating a compact and transferable library of causal knowledge.
We introduce a three-level hierarchical framework to formalize this idea. Low-level Executors and mid-level Controllers learn context-specific predictive world models for motor control and skill execution. At the highest level, a Curator uses counterfactual reasoning over imagined tasks to maintain causal models and skills that are most likely to be useful for future generalization. This framework recasts agent intelligence, instead of "prediction", but as a proactive "curation" of causal knowledge, leading to more resource-efficient and robust generalization.

Submission Type: Position/Review Paper (4-9 Pages)

Submission Number: 104