Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents

ICLR 2026 Conference Submission21443 Authors

Keywords: program synthesis, neurosymbolic systems, large language model agents, human-inspired-ai, planning

TL;DR: We enable abstraction learning in LLM-based program-synthesis agents.

Abstract:

Humans learn abstractions and use them to plan efficiently to quickly generalize across tasks---an ability that remains challenging for state-of-the-art large language model (LLM) agents and deep reinforcement learning (RL) systems.
Inspired by the cognitive science of how people form abstractions and intuitive theories of their world knowledge, Theory-Based RL (TBRL) systems, such as TheoryCoder, exhibit strong generalization through effective use of abstractions. However, they heavily rely on human-provided abstractions and sidestep the abstraction-learning problem.
We introduce TheoryCoder-2, a new TBRL agent that leverages LLMs' in-context learning ability to actively learn reusable abstractions rather than relying on hand-specified ones, by synthesizing abstractions from experience and integrating them into a hierarchical planning process. We conduct experiments on diverse environments, including BabyAI and VGDL games like Sokoban.
We find that TheoryCoder-2 is significantly more sample-efficient than baseline LLM agents augmented with classical planning domain construction, reasoning-based planning, and prior program-synthesis agents such as WorldCoder. TheoryCoder-2 is able to solve complex tasks that the baselines fail, while only requiring minimal human prompts, unlike prior TBRL systems.

Primary Area: neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)

Submission Number: 21443