Probabilistically Correct Language-Based Multi-Robot Planning Using Conformal Prediction

Jun Wang, Guocheng He, Yiannis Kantaros

Abstract: This paper addresses task planning problems for language-instructed robot teams. Tasks are expressed in natural language (NL), requiring the robots to apply their skills at various locations and semantic objects. Several recent works have addressed similar planning problems by leveraging pre-trained Large Language Models (LLMs) to design effective multi-robot plans. However, these approaches lack performance guarantees. To address this challenge, we introduce a new distributed LLM-based planner, called S-ATLAS for Safe plAnning for Teams of Language-instructed AgentS, that can achieve user-defined mission success rates. This is accomplished by leveraging conformal prediction (CP), a distribution-free uncertainty quantification tool. CP allows the proposed multi-robot planner to reason about its inherent uncertainty, due to imperfections of LLMs, in a distributed fashion, enabling robots to make local decisions when they are sufficiently confident and seek help otherwise. We show, both theoretically and empirically, that the proposed planner can achieve user-specified task success rates, assuming successful plan execution, while minimizing the average number of help requests. We provide comparative experiments against related works showing that our method is significantly more computational efficient and achieves lower help rates.