FastRLAP: A System for Learning High-Speed Driving via Deep RL and Autonomous Practicing

Dhruv Shah, Kyle Stachowicz, Arjun Bhorkar, Ilya Kostrikov, Sergey Levine

Keywords: robot learning, vision-based navigation, autonomous learning

Abstract: We present a system that enables an autonomous small-scale rally car to drive at high speed from only image observations using reinforcement learning. Our method trains autonomously in the real world, without human interventions, and without requiring any simulation. Our system integrates a number of important components to make this possible: we initialize the representations for the RL policy and value function from a large prior dataset of other robots navigating in other environments (at low speed), which provides initial navigationally relevant representations. From here, a sample-efficient online RL method uses a single low-speed user-provided demonstration to determine the desired driving course, extracts a set of navigational checkpoints, and autonomously practices driving through these checkpoints, resetting automatically on collision or failure. Perhaps surprisingly, we find that with appropriate initialization and choice of algorithm, our system can learn to drive over a variety of racing courses with just 10-20 minutes of online training. The resulting policies exhibit emergent aggressive driving skills, such as timing breaking and acceleration around turns, and match or exceed the lap times of a human driver using a similar first-person interface.