Structured Preference Optimization for Vision-Language Long-Horizon Task Planning

ACL ARR 2025 February Submission5006 Authors

Abstract:

Existing methods for vision-language task planning excel in short-horizon tasks but often fall short in complex, long-horizon planning within dynamic environments. These challenges primarily arise from the difficulty of effectively training models to produce high-quality reasoning processes for long-horizon tasks. To address this, we propose \textbf{S}tructured \textbf{P}reference \textbf{O}ptimization (\textbf{SPO}), which aims to enhance reasoning and action selection in long-horizon task planning through structured preference evaluation and optimized training strategies.
Specifically, SPO introduces:
1) Preference-Based Scoring and Optimization, which systematically evaluates reasoning chains based on task relevance, visual grounding, and historical consistency;
and 2) Curriculum-Guided Training, where the model progressively adapts from simple to complex tasks, improving its generalization ability in long-horizon scenarios and enhancing reasoning robustness.
To advance research in vision-language long-horizon task planning, we introduce ExtendaBench, a comprehensive benchmark covering 1,509 tasks across VirtualHome and Habitat 2.0, categorized into ultra-short, short, medium, and long tasks. Experimental results demonstrate that SPO significantly improves reasoning quality and final decision accuracy, outperforming prior methods on long-horizon tasks and underscoring the effectiveness of preference-driven optimization in vision-language task planning.
Specifically, SPO achieves a +5.98\% GCR and +4.68\% SR improvement in VirtualHome and a +3.30\% GCR and +2.11\% SR improvement in Habitat over the best-performing baselines.

Paper Type: Long

Research Area: Multimodality and Language Grounding to Vision, Robotics and Beyond

Research Area Keywords: Task Planning; Multimodal Large Language Model; Reasoning

Contribution Types: Model analysis & interpretability, Data resources

Languages Studied: English

Submission Number: 5006