Factored State Sampling

Matan Argaman, Guy Azran, Sarah Keren

Keywords: Task Planning; Generalized Task Planning; VLM; LLM;

TL;DR: We present the Factored State Sampler (FSS), a new domain-knowledge-based approach that significantly improves state estimation validity, advancing generalization in task planning.

Abstract:

Task planning requires accurate state estimation to achieve goals, but current methods rely on manually created, domain-specific functions that are time-consuming and struggle to adapt. Recent advances in Large Language Models (LLMs) and Vision-Language Models (VLMs) enable zero-shot semantic state estimation but often lack task-specific accuracy, leading to suboptimal plans or goal failure. To address this, we propose the Factored State Sampler (FSS), which integrates task domain knowledge to refine state estimation.
While the FSS slightly reduces the micro-average AUC on state features, it substantially enhances state validity, which is a critical metric for effective task planning. The highest state validity is achieved by combining an additional neural network with the FSS, demonstrating the significant impact of our approach on enhancing generalization in task planning.

Submission Number: 39