Instance Segmentation of Point Clouds: from City Roads to Forests

Binbin Xiang

Abstract: Remote sensing technology enables large-scale observation of the Earth, serving as a basis for planning and management. To obtain fine-scale and object-level information in urban and forest environments, researchers are developing remote sensing tools and data processing methods capable of capturing more detailed information. Among them, point cloud data obtained through LiDAR technology is an effective means to acquire high precision three-dimensional information. However, the unordered and discrete nature of point cloud data, coupled with the large data volume in large-scale outdoor scenes, increase the challenge and difficulty of processing. This thesis first reviews the development and application of three-dimensional point cloud deep learning networks, compares various three-dimensional backbone networks and instance segmentation strategies through extensive experiments, and proposes an effective point cloud panoptic segmentation pipeline for outdoor large-scale scenes. Subsequently, this pipeline is applied to two important types of outdoor scenes: city roads captured with mobile laser scanning systems and forest environments captured with airborne laser scanners. A focus is put on correctly segmenting closely located instances of the same class. Finally, focusing on the particularly challenging forest scenes, the pipeline is extended into an integrated system for multiple segmentation tasks, including individual tree segmentation, semantic segmentation of scene components, and tree component segmentation. Based on the segmentation, it becomes possible to automatically predict various useful quantities for a forest inventory: individual tree-level features and stand-wise attributes. In summary, the thesis contributes to the progress of point cloud instance segmentation in city roads and forest scenes, in support of applications such as 3D urban mapping and automated and smart management of forest ecosystems.