Learning Rewards and Skills to Follow Commands with a Data Efficient Visual-Audio Representation

Peixin Chang, Shuijing Liu, Tianchen Ji, Neeloy Chakraborty, D Livingston McPherson, Katherine Rose Driggs-Campbell

Keywords: Robotics, Representation Learning, Reinforcement Learning

TL;DR: We learn a representation to generate a reward function to train command-following robots with reinforcement learning

Abstract: Based on the recent advancements in representation learning, we propose a novel framework for command-following robots with raw sensor inputs. Previous RL-based methods are either difficult to continuously improve after the deployment or require a large number of new labels during the fine-tuning. Motivated by (self-)supervised contrastive learning literature, we propose a novel representation, named VAR++, that generates an intrinsic reward function for command-following robot tasks by associating images with sound commands. After the robot is deployed in a new domain, the representation can be updated intuitively and data-efficiently by non-expert, and the robots are able to fulfill sound commands without any hand-crafted reward functions. We demonstrate our approach to various sound types and robotic tasks, including navigation and manipulation with raw sensor inputs. In the simulated experiments, we show that our system can continually self-improve in previously unseen scenarios given fewer new labeled data, yet achieves better performance, compared with previous methods.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Reinforcement Learning (eg, decision and control, planning, hierarchical RL, robotics)