Reinforced Reasoning for Interactive Multi-step Embodied Planning

Di Wu, Jiaxin Fan, Junzhe Zang, Guanbo Wang, Wei Yin, Wenhao Li, Bo Jin

Keywords: Reinforcement Fine-Tuning, Embodied Planning, Vision Language Models

Abstract:

Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and natural language goals.
While recent vision-language models (VLMs) excel at static perception tasks, they struggle in interactive environments.
In this work, we introduce a reinforcement fine-tuning framework that brings R1-style reasoning enhancement into embodied planning.
We adopt an offline reward paradigm to avoid costly online interaction, design a rule-based reward function tailored to multi-step action quality and optimize the policy via Generalized Reinforced Preference Optimization (GRPO).
Our approach is evaluated on Embench, a recent benchmark for interactive embodied tasks, covering both in-domain and out-of-domain scenarios.
Experimental results show that our method significantly outperforms models of similar or larger scale, including GPT-4o-mini and 70B+ open-source baselines, and exhibits strong generalization to unseen environments. This work highlights the potential of reinforcement-driven reasoning to advance multi-step planning in embodied AI.

Submission Number: 98