NBV/NBC Planning Considering Confidence Obtained From Shape Completion Learning

Ruishuang Liu, Chuan Li, Weiwei Wan, Jia Pan, Kensuke Harada

Abstract: In this letter, we present a novel approach for planning an object's Next Best Views (NBV) so that a depth camera can collect the object's surface point cloud and reconstruct its 3D model with a small number of consequent views. Our focus is especially on thin and curved metal plates, and we use a robot manipulator and an externally installed stationary depth sensor as the experimental system. The targeted objects have shiny and flat surfaces, which leads to noisy point cloud data and low guidance in the surface normal for completion. To overcome these challenges, we propose using a Point cloud Completion Network (PCN) to find heuristics for NBV (when the depth sensor is mounted on a robot's end flange) or Next Best robot Configuration (NBC, when the depth sensor is externally fixed) optimization. Unlike previous methods, our approach predicts NBV by considering a holistic view of the object predicted by neural networks, which is not limited by the local information captured by the sensors and is, therefore, robust to deficiencies in known point cloud data and normal. We conducted simulation and real-world experiments to evaluate the proposed method's performance. Results show that the proposed method efficiently solves the NBV problems and can satisfactorily model thin and curved metal plates.