Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human VideosDownload PDF

Anonymous

Published: 15 Jun 2022, Last Modified: 05 May 2023SSL-RL 2021 OralReaders: Everyone
Keywords: Deep Reinforcement Learning, Robot Learning, Multi-Task Learning
TL;DR: Using "In-The-Wild" videos of humans to learn generalizable reward functions for robot learning.
Abstract: We are motivated by the goal of generalist robotic agents that can complete a wide range of tasks across many environments. Critical to this is the robot’s ability to acquire some metric of task success or reward, which is necessary for reinforcement learning, planning, or knowing when to ask for help. For a general-purpose robot operating in the real world, this reward function must also be able to generalize broadly across environments, tasks, and objects, while depending only on-board sensor observations (e.g. RGB images). While deep learning on large and diverse datasets has shown promise as a path towards such generalization in computer vision and natural language, collecting high quality data of robotic interaction at scale remains an open challenge. In contrast, “in-the-wild” videos of humans (e.g. YouTube) contain an extensive collection of people doing interesting tasks across a diverse range of settings. Unlocking this data for learning robotic reward functions could lead to more scalable training of robots for numerous tasks and would be an important step towards generalist robots. In this work, we propose a simple approach, Domain-agnostic Video Discriminator (DVD), for learning multitask reward functions which can generalize by virtue of learning from a small amount of robot data with a broad dataset of human videos. We find that by leveraging diverse human datasets, this reward function (a) can generalize zero shot to unseen environments, (b) generalize zero shot to unseen tasks, and (c) can be combined with visual model predictive control to solve robotic manipulation tasks on a real WidowX200 robot in an unseen environment from a single human demo.
0 Replies

Loading