Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections

Lihan Zha, Yuchen Cui, Li-Heng Lin, Minae Kwon, Montserrat Gonzalez Arenas, Andy Zeng, Fei Xia, Dorsa Sadigh

Keywords: Learning from language corrections, incremental learning

TL;DR: We propose a LLM-based method for robots to effectively respond and remember online language corrections.

Abstract:

Today's robot policies exhibit subpar performance when faced with the challenge of generalizing to novel environments. Therefore, adapting to and learning from online human corrections is essential but a non-trivial endeavor: not only do robots need to remember human feedback over time to retrieve the right information in new settings and reduce the intervention rate, but also they would need to be able to respond to feedback that can take arbitrary corrections about high-level human preferences to low-level adjustments to skill parameters.
In this work, we present Distillation and Retrieval of Online Corrections (DROC), an LLM-based system that can respond to arbitrary forms of language feedback, distill generalizable knowledge from corrections, and retrieve relevant past experiences based on textual and visual similarity for improving performance in novel settings. DROC is able to respond to a sequence of online language corrections that address failures in both high-level task plans and low-level skill primitives. We demonstrate DROC effectively distills the relevant information from the sequence of online corrections in a knowledge base and retrieves that knowledge in settings with new task or object instances. DROC outperforms baseline Code as Policies by using only half of the total number of corrections needed in the first round and requires little to no corrections after two iterations.

Submission Number: 7