Closed-Loop Long-Horizon Robotic Planning via Equilibrium Sequence Modeling

Jinghan Li, Zhicheng Sun, Yadong MU

TL;DR: An equilibrium model-based LLM planner capable of self-refining plans based on external and internal feedback.

Abstract: In the endeavor to make autonomous robots take actions, task planning is a major challenge that requires translating high-level task descriptions to long-horizon action sequences. Despite recent advances in language model agents, they remain prone to planning errors and limited in their ability to plan ahead. To address these limitations in robotic planning, we advocate a self-refining scheme that iteratively refines a draft plan until an equilibrium is reached. Remarkably, this process can be optimized end-to-end from an analytical perspective without the need to curate additional verifiers or reward models, allowing us to train self-refining planners in a simple supervised learning fashion. Meanwhile, a nested equilibrium sequence modeling procedure is devised for efficient closed-loop planning that incorporates useful feedback from the environment (or an internal world model). Our method is evaluated on the VirtualHome-Env benchmark, showing advanced performance with improved scaling w.r.t. inference-time computation. Code is available at https://github.com/anonymous-icml-2025/equilibrium-planner.

Lay Summary:

Task planning is a major challenge in robotics, which requires translating high-level task descriptions to long-horizon action sequences. For example, "Get me a glass of juice" needs to be decomposed into "Walk To Refrigerator, Open Refrigerator, Pick Up Juice, Close Refrigerator, Put Juice On Table". Current large language model(LLM)-based methods remain unsatisfactory. We hope to improve the performance of LLMs in task planning.

We advocate training LLM planners to self-refine, enabling them to achieve better results by correcting themselves. However, such training is costly and complex using traditional methods. We propose a new method named equilibrium sequence modeling, which can be optimized in a simple supervised learning fashion. This allows us to train self-refining plannes easily. Meanwhile, we design mutiple components to process feedback and achieve closed-loop planning. Our method performed well in experiments.

Our method has implications for exploring the planning capabilities of LLMs. It is a novel approach that encourages LLMs to think more deeply and plan ahead.

Link To Code: https://github.com/Singularity0104/equilibrium-planner

Primary Area: Applications->Robotics

Keywords: Deep Equilibrium Models, Large Language Models, Robot Task Planning

Submission Number: 1153