Lifelong Autonomous Improvement of Navigation Foundation Models in the Wild

Kyle Stachowicz, Lydia Ignatova, Sergey Levine

Keywords: Navigation, Reinforcement Learning, Lifelong Learning

TL;DR: We propose a framework for autonomously improving offline RL navigation policies with online RL with minimal-to-no human supervision

Abstract: Recent works have proposed a number of general-purpose robotic foundation models that can control a variety of robotic platforms to perform a range of different tasks, including in the domains of navigation and manipulation. However, such models are typically trained via imitation learning, which precludes the ability to improve autonomously through experience that the robot gathers on the job. In this work, our aim is to train general-purpose robotic foundation models in the domain of robotic navigation specifically with the aim of enabling autonomous self-improvement. We show that a combination of pretraining with offline reinforcement learning and a complete system for continual autonomous operation leads to a robotic learning framework that not only starts off with broad and diverse capabilities, but can further improve and adapt those capabilities in the course of carrying out navigational tasks in a given deployment location. To our knowledge, our model LiReN is the first navigation robot foundation model that is capable of fine-tuning with autonomous online data in open-world settings.

Video: https://bit.ly/liren-video-corl2024

Website: https://kylestach.github.io/lifelong-nav-rl/

Code: https://github.com/kylestach/lifelong-nav-rl

Student Paper: yes

Submission Number: 659