Object-centric Representations for Interactive Online Learning with Non-Parametric Methods

Nikhil Uday Shinde, Jacob John Johnson, Sylvia Lee Herbert, Michael C. Yip

Abstract:

Abstract— Large offline learning-based models have enabled
robots to successfully interact with objects for a wide variety of
tasks. However, these models rely on fairly consistent structured
environments. For more unstructured environments, an online
learning component is necessary to gather and estimate information about objects in the environment in order to successfully
interact with them. Unfortunately, online learning methods like
Bayesian non-parametric models struggle with changes in the
environment, which is often the desired outcome of interactionbased tasks. We propose using an object-centric representation
for interactive online learning. This representation is generated
by transforming the robot’s actions into the object’s coordinate
frame. We demonstrate how switching to this task-relevant
space improves our ability to reason with the training data
collected online, enabling scalable online learning of robotobject interactions. We showcase our method by successfully
navigating a manipulator arm through an environment with
multiple unknown objects without violating interaction-based
constraints.