Learning Task-Informed Exploration Policies for Active System Identification

Marina Y. Aoyama, Joao Moura, Juan Del Aguila Ferrandis, Sethu Vijayakumar

Keywords: reinforcement learning, interactive perception, system identification, manipulation

TL;DR: We propose a task-informed exploration approach, based on reinforcement learning, that learns to explore and perform dynamic tasks with objects of unknown physical properties, and validate it in both simulation and on a physical KUKA iiwa robot arm.

Abstract: In many dynamic robotic tasks, such as striking pucks into a goal outside the reachable workspace, the robot must first identify the relevant physical properties of the object for successful task execution, as it is unable to recover from failure or retry without human intervention. To address this challenge, we propose a task-informed exploration approach, based on reinforcement learning (RL), that trains an exploration policy using rewards automatically generated from the sensitivity of a privileged task policy to errors in estimated properties. We also introduce an uncertainty-based mechanism to determine when to transition from exploration to task execution, ensuring sufficient property estimation accuracy with minimal exploration time. Our method achieves a 90\% success rate on the striking task---significantly outperforming baselines that achieve at most 40\% success. Additionally, we demonstrate that our task-informed exploration rewards capture the relative importance of physical properties. Finally, we validate our approach on two manipulation tasks in a physical setup.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 18