Benchmarking Object-Centric Manipulation Using a Simulated Environment

Hao Zhu

Abstract:

Benchmark is of vital importance to evaluate the effectiveness of an algorithm. In recent years, the computer vision community benefits from using standard datasets and benchmarks. This trend is becoming more popular in the robotics community as well.
However, if we would like to bring the same comparing approach to the robotics community, there are some challenges we have to face. Is there an alternative way to benchmark the performance of robots? We think using a simulated environment is the way out.
Humans study the physical world from the interaction with objects, and we believe robots should study in the same way. As building home assistant robots is our final goal, we started from building a dataset of articulated objects for robots to interact with.
Finally, with the help of the SAPIEN robotics kit that supports URDF and ROS, researchers have the possibility to train and test the robot in the simulator, and transfer it to the real world.
This paper also discusses the opportunities for improvement of the original paper, and proposes the new perspectives and future works since publishing.

TL;DR: SAPIEN is a realistic and physics-rich simulated environment that hosts a large-scale set for articulated objects. It enables various robotic vision and interaction tasks that require detailed part-level understanding.

Keywords: manipulation, benchmark, simulator