Household Task Planning with Multi-Objects State and Relationship Using Large Language Models Based Preconditions Verification

Jin Aoyama, Sudesna Chakraborty, Takeshi Morita, Shusaku Egami, Takanori Ugai, Ken Fukuda

Abstract: We propose a novel approach to household task planning that leverages Large Language Models (LLMs) to comprehend and consider environmental states. Unlike previous methods that depend primarily on commonsense reasoning or visual inputs, our approach focuses on understanding object states and relationships within the environment. To evaluate the capability, we developed a specialized dataset of household tasks that specifically tests LLMs’ ability to reason about object states, identifiers, and relationships. Our method combines simulator-derived environmental state information with an LLM-based planning to generate executable action sequences. A key feature in our system is the LLM-driven verification mechanism that assesses whether environmental preconditions are met before each action executes, automatically reformulating action steps when prerequisites are not satisfied. Experimental results using GPT-4o demonstrate strong performance, achieving 89.4% success rate on state change

External IDs:dblp:conf/icaart/AoyamaC0EUF25