Learning Surface Reconstruction from Point Clouds in the Wild. (Reconstruction de surfaces à partir de nuages de points par apprentissage profond)

Raphael Sulzer

Abstract: Digital representations of the physical world allow to replace costly and time-consuming experiments in the field by efficient numerical simulations on a computer. Creating digital representation of the physical world involves the transition between physical measurements and mathematical and digital models and is a long standing problem in digital geometry processing.In this thesis, we investigate the problem of surface reconstruction from point clouds in the wild, using novel deep learning methods. Point clouds in the wild are generated from measurements acquired outside of the laboratory, either directly by 3D scanners or indirectly from 2D photogrammetric images. They often include defects such as noise, outliers, non-uniform sampling or missing data, which complicate the reconstruction of a topologically and geometrically accurate surface. Furthermore, they can depict large scenes with a multitude of different objects and clutter. This makes the problem of surface reconstruction particularly challenging for deep learning methods, which have otherwise shown great success for solving problems in related tasks. Supervised machine learning e.g. has become a powerful tool when adequate training data for the learning algorithm is available. However, such training data is difficult to gather for the task of surface reconstruction of real-world objects or scenes. Moreover, existing learning architectures may also not be suited for processing large-scale point clouds with millions of points. In this thesis, we alleviate some of these issues by introducing novel, learning-based methods that can handle large-scale point clouds with real-world characteristics while being trained on small synthetic datasets. The thesis includes three main contributions.First, we survey and benchmark several learning-free and learning-based methods that address the problem of surface reconstruction from point clouds, for which a variety of methods have been proposed over the last three decades. Pre-learning methods often propose using different prior assumptions either on the input point cloud or output surface, to make the problem tractable, and produce satisfying results even under challenging conditions. In contrast, recently introduced Deep Surface Reconstruction (DSR) algorithms can learn directly from the data and therefore promise to reconstruct more accurate surfaces than previously possible. Our findings show that, while DSR methods are well suited for learning sampling related data priors from a given training set of point clouds and corresponding surfaces, a few classical methods still perform better, mainly in terms robustness to point cloud defects.Another shortcoming of most DSR methods is the fact that they ignore sensor poses and only operate on point locations. However, sensor visibility holds meaningful information regarding space occupancy and surface orientation. Therefore, we present two simple ways to augment point clouds with visibility information, so it can directly be leveraged by surface reconstruction networks with minimal adaptation. Our proposed modifications consistently improve the accuracy of generated surfaces as well as the generalization capability of the networks to unseen domains.Lastly, we introduce a novel learning-based, visibility-aware, surface reconstruction method for large-scale, defect-laden point clouds. Our approach can cope with the scale and variety of point cloud defects encountered in real-life acquisitions. Our method relies on a 3D Delaunay tetrahedralization (3DT) whose cells are classified as inside or outside the surface by a graph neural network and an energy model solvable with a graph cut. Our approach, making use of both local geometric attributes and line-of-sight visibility information, is able to learn a visibility model from a small amount of synthetic training data and generalize to real-life acquisitions