Lifelong Experience Abstraction and Planning

Peiqi Liu, Leslie Pack Kaelbling, Joshua B. Tenenbaum, Jiayuan Mao

Keywords: Continual Learning, Human-in-the-Loop, Long-Horizon Tasks Planning, Embodied Planning, Large Language Models, Benchmarking

TL;DR: A framework for continual behavior learning in embodied agents through interaction with the environment and guidance from humans

Track: Long Paper (up to 9 pages)

Abstract: We present LEAP (Lifelong Experience Abstraction and Planning), a framework for continual behavior learning in embodied agents through interaction with the environment and guidance from humans. LEAP addresses the challenge of representing flexible knowledge about tasks and environments --- ranging from constraints and subgoal sequences to action plans and high-level goals --- in a unified framework. At its core, LEAP builds on the Crow Definition Language (CDL), a behavior rule language that integrates imperative programming with declarative planning by allowing agents to express both executable subroutines and subgoal hierarchies. Leveraging large language models (LLMs), LEAP translates diverse human instructions into CDL programs, generates planning-compatible code, and abstracts reusable behavior rules from successful executions to support future generalization. LEAP maintains a library of such CDL programs, enabling the agent to accumulate and refine its behavioral repertoire over time. We evaluate LEAP on the VirtualHome benchmark, demonstrating its ability to represent a wide variety of human instructions and its capacity to continually improve task performance through experience and interaction.

Format: We have read the camera-ready instructions, and our paper is formatted with the provided template.

De-Anonymization: This submission has been de-anonymized.

Anonymization: This submission has been anonymized for double-blind review via the removal of identifying information such as names, affiliations, and identifying URLs.

Submission Number: 19