Quantitative and flexible 3D shape dataset augmentation via latent space embedding and deformation learning

Jiarui Liu, Qing Xia, Shuai Li, Aimin Hao, Hong Qin

Abstract: Deep learning techniques for geometric processing have been gaining popularity in recent years, various deep models (i.e., deep learning methods based on neural networks) are developed with enhanced performance and functionality in conventional geometric tasks such as shape classification, segmentation, and recognition. Yet, deep models would rely on large datasets for the training and testing purpose, which are generally lacking as 3D shape geometry could not be easily acquired and/or reconstructed. In this paper, we propose a new 3D shape dataset augmentation method by learning the deformation between shapes in a highly reduced latent space while affording interactive control of shape generation. Specifically, we model each shape using a concise skeleton-based representation, and then we apply Gaussian Process Latent Variable Model (GPLVM) to embed all shape skeletons into a low-dimensional latent space, where new skeletons could be generated with diverse kinds of flexible control and/or quantitative guidance. A second network that learns the displacement between shapes can be employed to produce new 3D shape from newly-generated skeletons. Compared with popular computer vision techniques, our new generative method could overcome remaining challenges of 3D shape augmentation with new characteristics. Specifically, our new method is capable of transforming 3D shapes in a more liberal way, preserving their geometric properties at a semantic level, and creating new shape with ease and flexible control. Extensive experiments have exhibited the capability and flexibility of our new method in generating new shapes using only few samples. Our shape augmentation is an effective way to simultaneously improve the shape creation capability and the shape extrapolation accuracy, and it is also of immediate benefit to almost all deep learning tasks in geometric modeling and processing.