FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learning
Keywords: manipulation, imitation learning, benchmarking
TL;DR: A benchmark for functional manipulation together with open-sourced dataset and object CAD files
Abstract: In this paper, we propose a benchmark for studying robotic learning for functional manipulation. We identify handling complex contact dynamics and generalization as two central challenges in practical robotic manipulation. While many prior works have addressed one challenge or the other, few have studied both in combination. We hypothesize that making progress on the combination of these challenges requires a set of real-world benchmark tasks that balance complexity with accessibility, providing a set of tasks that are sufficiently narrowly scoped that models and datasets of reasonable scale can be used to make progress, but sufficiently varied that they present a meaningful generalization challenge not just in terms of basic and imprecise skills such as grasping, but also more complex and precise behaviors that require functional manipulation, such as repositioning and reorienting an object for a precise assembly task. Our functional manipulation benchmark consists of a variety of 3D printed objects that can be reproduced precisely by other researchers, each one requiring a sequence of grasping, reorientation, and assembly behaviors. Generalization can be evaluated on test objects and varied positions, as well as more complex multi-stage assembly tasks. We also provide an imitation learning system that provides a basic set of policies for each skill, allowing researchers to use our tasks as a toolkit for studying any portion of the pipeline -- for example by proposing a better design for a grasping controller and evaluating it in combination with our baseline reorientation and assembly controllers. Our dataset, object CAD files and evaluation videos can be found on our project website: https://sites.google.com/view/manipulationbenchmark
Submission Number: 1
Loading