PoE-World: Compositional World Modeling with Products of Programmatic Experts

Wasu Top Piriyakulkij, Yichao Liang, Hao Tang, Adrian Weller, Marta Kryven, Kevin Ellis

Keywords: program synthesis, compositionality, world modeling

TL;DR: We introduce a novel program synthesis approach to output world models of complex, non-gridworld domains by representing world models as products of programmatic experts.

Abstract:

Learning how the world works is central to building AI agents that can adapt to complex environments.
Traditional world models based on deep-learning demand vast amounts of training data, and do not flexibly update their knowledge from sparse observations.
Recent advances in program synthesis using Large Language Models (LLMs) give an alternate approach which learns world models represented as source code, supporting strong generalization from little data.
To date, application of program-structured world models remains limited to natural language and grid-world domains. We introduce a novel program synthesis method for effectively modeling complex, non-gridworld domains by representing a world model as an exponentially-weighted product of programmatic experts (PoE-World) synthesized by LLMs.
We show that this approach can learn complex, stochastic world models from just a few observations.
We evaluate the learned world models by embedding them in a model-based planning agent, demonstrating efficient performance and generalization to unseen levels on Atari's Pong and Montezuma's Revenge.

Primary Area: General machine learning (supervised, unsupervised, online, active, etc.)

Submission Number: 7683