Continual Robot Learning via Language-Guided Skill Acquisition

Shuo Cheng, Zhaoyi Li, Kelin Yu, Danfei Xu

Keywords: Reinforcement Learning, Task and Motion Planning, Large Language Model, Continual Learning

Abstract: To support daily human tasks, robots need to tackle complex, long-horizon tasks and continuously acquire new skills to handle new problems. Deep reinforcement learning (DRL) offers the potential to learn fine-grained skills, but is based heavily on human-defined rewards and faces challenges with long-term goals. Task and Motion Planning (TAMP) is adept at handling long-horizon tasks but often needs tailored domain-specific skills, resulting in practical limitations and inefficiencies. To address these challenges, we propose LG- SAIL (Language Models Guided Sequential, Adaptive, and Incremental Skill Learning), a framework that leverages Large Language Models (LLMs) to synergistically integrate TAMP and DRL for continuous skill learning in long-horizon tasks. Our framework achieves automatic task decomposition, skill creation, and dense reward generation to efficiently acquire the desired skills. To facilitate new skill learning, our framework maintains a symbolic skill library and utilizes the existing model from semantic-related skills to warm start the training. LG-SAIL demonstrates superior performance compared to baselines across six challenging simulated task domains across two benchmarks. Furthermore, we demonstrate the ability to reuse learned skills to expedite learning in new task domains, and deploy the system on a physical robot platform.

Submission Number: 17