A Cuboid Detection and Tracking System using A Multi RGBD Camera Setup for Intelligent Manipulation and Logistics

Haohao Hu, Fabian Immel, Johannes Janosovits, Martin Lauer, Christoph Stiller

Abstract: Cuboid objects are widely used in logistics and manufacturing, since they are ideal for automatic manipulation. For that, a reliable detection, identification and tracking algorithm of cuboid objects is a key requirement. RGBD cameras are suited for these tasks, however they suffer from limitations such as occlusions from clutter or a small field of view. A multi RGBD camera setup would remedy many of these limitations. In this work, we present a cuboid detection, identification and tracking system using a multi RGBD camera setup. First, we calibrate our multi camera setup using a robot arm and a colored spherical calibration target. By using spatial and color information, a preprocessing pipeline segments then the unified point cloud into locally connected planar segments. Afterwards, custom spatial criteria allows for assigning the resulting segments to cuboid groups. We present a new modified bounding box regression method to estimate cuboid boxes and a novel intersection over union based approach to track cuboids across frames. By applying our approach, the automatic manipulation and transportation in manufacturing industry will become more efficient, which helps to save more time and cost. Our system is tested and analyzed with a recorded dataset, which contents variant constellations. The reliability and accuracy are demonstrated.