Interactive Task Planning with Language Models

Boyi Li, Philipp Wu, Pieter Abbeel, Jitendra Malik

Keywords: Task Planning, Large Language Models

TL;DR: We present Interactive Task Planning, a framework for connecting LLMs with robots to allows humans to not only provide high level commands to robots in an interactive manner.

Abstract: An interactive robot framework accomplishes long-horizon task planning and can easily generalize to new goals or distinct tasks, even during execution. However, most traditional methods require predefined module design, which makes it hard to generalize to different goals. Recent large language model based approaches can be more open-ended but often require heavy prompt engineering or domain specific pretrained models. To tackle this problem, we propose an efficient framework that achieves interactive task planning with language models. Our system interacts with both high-level plans and low-level functions via language inputs. We verify the robustness of our system in generating novel high-level instructions for unseen objectives and its ease of adaptation to different tasks by merely substituting the task guidelines, without the need for additional complex prompt engineering. Furthermore, when the user sends a new request, our system is able to replan accordingly with precision based on the new request, task guidelines and previously executed steps.

Submission Number: 23