Leveraging Large Language Models to Repair High-level Robot Controllers from Assumption Violations

Qian Meng, Jin Peng Zhou, Kilian Q Weinberger, Hadas Kress-Gazit

Keywords: Formal methods; Robotics; large language models

TL;DR: A hybrid approach that combines large language models (LLMs) and formal methods to speed up runtime repair of high-level robot controllers from assumption violations.

Abstract: This paper presents INPROVF, an automatic framework that combines large language models (LLMs) and formal methods to speed up the repair process of high-level robot controllers. Previous approaches based solely on formal methods are computationally expensive and cannot scale to large state spaces. In contrast, INPROVF uses LLMs to generate repair candidates, and formal methods to verify their correctness. To improve the quality of these candidates, our framework first translates the symbolic representations of the environment and controllers into natural language descriptions. If a candidate fails the verification, INPROVF provides feedback on potential unsafe behaviors or unsatisfied tasks, and iteratively prompts LLMs to generate improved solutions. We demonstrate INPROVF through 12 violations with various workspaces, tasks, and state space sizes.

Submission Number: 43