Conformal Temporal Logic Planning using Large Language Models

Jun Wang, Jiaming Tong, Kaiyuan Tan, Yevgeniy Vorobeychik, Yiannis Kantaros

Abstract:

This paper addresses a novel task planning problem for mobile robots with missions requiring the accomplishment of multiple high-level sub-tasks in a temporal and logical order, where the sub-tasks are expressed using natural language (NL). To formally define the mission, we treat these NL-specified sub-tasks as atomic predicates in a Linear Temporal Logic (LTL) formula. We refer to this novel framework for formal task specification as LTL-NL. Our goal is to design robot plans accomplishing LTL-NL tasks, a problem that cannot be solved directly by existing LTL planners because of the NL nature of atomic propositions defining the sub-tasks. To address this problem, we propose HERACLEs, a hierarchical neurosymbolic planner that relies on a novel integration of (i) symbolic temporal logic planners generating high-level task plans determining what NL-based sub-tasks should be accomplished and in what order to successfully complete the mission; (ii) Large Language Models (LLMs) to design robot plans, defined as sequences of robot actions, implementing the task plans; and (iii) conformal prediction (CP) acting as a formal interface between (i) and (ii). CP allows the proposed planner to reason
about the uncertainty of the employed LLM in its outputs. In cases of high uncertainty, the LLM asks for help from the symbolic planner to revise the task plan and, if unsuccessful, from users. We show, both theoretically and empirically, that HERACLEs can achieve high mission success rates due to this conformal interface. We provide extensive comparative experiments demonstrating that HERACLEs outperforms state-of-the-art LLM-based planning approaches in terms of its ability to design correct plans. The advantage of HERACLEs over baselines grows as the mission complexity increases.