Obstacle detection by multi-sensor fusion of a laser scanner and depth camera

Zainab Saleem, Philip Long, Saif Huq, Marion McAfee

Abstract: Reliable sensor systems are essential to detect and track the human operator in a human-robot collaborative environment. This work proposes a low-cost obstacle warning system by fusing 2D LiDAR and 3D vision data. Some regions of the workspace can be inaccessible to a single sensor, defined as blind spots. Therefore, by using a 2D LiDAR on the robot's base and a vision sensor on top of the workspace, we ensure coverage of the entire workspace of the manipulator. For a more efficient system, first human operator has been detected using an object detection algorithm then the laser points are segmented. Further, to obtain more accurate results, the data from both sensors has been fused using Kalman Filter. This fusion not only provides accurate and fast distance information on the position of a human worker without leaving any blind spots but is also significantly more affordable than the more common 3D LiDAR plus vision approach.