GraphPlan: Graph-enhanced Planning via Thinking LLMs for Embodied Agents

Chen Li, Sijie Cheng, Yuelin Zhang, Junxi LI, Yang Liu, Wenbing Huang

Keywords: Embodied Agent, Instruction Following, Task Planning, Task Graph, Reinforcement Learning

Abstract: Embodied agents that follow instructions to complete complex tasks in visual environments have attracted increasing attention. Large Language Models (LLMs) based planners, notwithstanding the progress achieved, still suffer from three main limitations: (i) a lack of physical grounding, often resulting in hallucinatory plans; (ii) poor generalization to unseen long-horizon tasks; and (iii) an absence of environmental awareness in the open-loop planning process. To address these issues, we propose GraphPlan, a novel framework that integrates a task graph to provide structured knowledge for robust planning and a scene graph to maintain environmental memory for event-driven replanning. Specifically, the task graph guides the LLM's reasoning through contextual prompting and iterative refinement, effectively mitigating planning hallucinations. Furthermore, within the GRPO framework, the task graph offers delicate reward design to train LLMs' reasoning, enhancing long-horizon planning capabilities and improving generalization. Finally, the memory constructed by a dynamic scene graph empowers an event-driven replanning module, enabling the agent to foster environment awareness and correct instruction misalignment within a closed-loop planning process. On the standard benchmark ALFRED, GraphPlan achieves state-of-the-art performance on the official leaderboard. Moreover, its high-level planner outperforms a series of leading API-based LLMs on both the validation set and unseen long-horizon tasks. Additional experiments reveal the promising potential of our graph-enhanced framework in few-shot or zero-shot learning scenarios, and its generalization to novel tasks beyond the benchmark.

Primary Area: applications to robotics, autonomy, planning

Submission Number: 8709