Physics-Based Selection of Informative Actions for Interactive Perception

Clemens Eppner, Roberto Martín-Martín, Oliver Brock

Abstract: Interactive perception exploits the correlation between forceful interactions and changes in the observed signals to extract task-relevant information from the sensor stream. Finding the most informative interactions to perceive complex objects, like articulated mechanisms, is challenging because the outcome of the interaction is difficult to predict. We propose a method to select the most informative action while deriving a model of articulated mechanisms that includes kinematic, geometric, and dynamic properties. Our method addresses the complexity of the action selection task based on two insights. First, we show that for a class of interactive perception methods, information gain can be approximated by the amount of motion induced in the mechanism. Second, we resort to physics simulations grounded in the real-world through interactive perception to predict possible action outcomes. Our method enables the robot to autonomously select actions for interactive perception that reveal most information, given the current knowledge of the world. This leads to improved perception and more accurate world models, finally enabling robust manipulation.