Learning 6DoF Grasp-poses in Occluded Scenes Using RGB and 3D Point Cloud Modalities

Abhay Dayal Mathur, Sachin Bhadang, Prem Raj, Laxmidhar Behera, Tushar Sandhan

Abstract: Automating the process of harvesting fruits in an orchard-like environment can be more economical than traditional hand-picking methods. The number of people working in the agricultural sector is reducing daily, increasing the cost of manual labour. Localisation and pose estimation of the target fruit is central to any automation-based solution for fruit harvesting. These tasks can be challenging due to the occlusion caused by branches and leaves, as it limits the options to approach the target fruit. The efficacy of deep learning methods using RGB and 3D point cloud data has been demonstrated recently for grasp-pose estimation in dynamic situations. However, this process requires a large amount of data labelled using application-specific annotation tools. Data augmentation is a powerful and cost-effective solution for reducing the requirement for labelled data. In this paper, we propose an augmentation method, 3D-CopyPaste (3DCP), for augmenting point-cloud data for estimating the Six Degrees of Freedom (6DoF) grasp pose. A 6DoF apple grasp-pose dataset and a two-step deep learning-based baseline for apple grasp-pose estimation are also analysed. Additionally, we present a setup for our customised semi-automated annotation method for labelling 6DoF apple grasp-poses, developed in conjunction with the RViz widget. All the code and tools will be made publicly available for future research.