Estimating Model Utility for Deformable Object Manipulation Using Multiarmed Bandit Methods

Dale McConachie, Dmitry Berenson

Abstract: We present a novel approach to deformable object manipulation that does not rely on highly accurate modeling. The key contribution of this paper is to formulate the task as a multiarmed bandit problem, with each arm representing a model of the deformable object. To “pull” an arm and evaluate its utility, we use the arm's model to generate a velocity command for the gripper(s) holding the object and execute it. As the task proceeds and the object deforms, the utility of each model can change. Our framework estimates these changes and balances the exploration of the model set with exploitation of high-utility models. We also propose an approach based on Kalman filtering for nonstationary multiarmed normal bandits to leverage the coupling between the models to learn more from each arm pull. We demonstrate that our method outperforms previous methods on synthetic trials and performs competitively on several manipulation tasks in simulation.