A Simple and Scalable Shape Representation for 3D Reconstruction

Mateusz Michalkiewicz, Eugene Belilovsky, Mahsa Baktashmotagh, Anders Eriksson

TL;DR: We show that a shape representation based on applying PCA to the signed distance transform can be effective for shape inference tasks.

Abstract: Deep learning applied to the reconstruction of 3D shapes has seen growing interest. A popular approach to 3D reconstruction and generation in recent years has been the CNN decoder-encoder model often applied in voxel space. However this often scales very poorly with the resolution limiting the effectiveness of these models. Several sophisticated alternatives for decoding to 3D shapes have been proposed typically relying on alternative deep learning architectures. We show however in this work that standard benchmarks in 3D reconstruction can be tackled with a surprisingly simple approach: a linear decoder obtained by principal component analysis on the signed distance transform of the surface. This approach allows easily scaling to larger resolutions. We show in multiple experiments it is competitive with state of the art methods and also allows the decoder to be fine-tuned on the target task using a loss designed for SDF transforms, obtaining further gains.

Keywords: Computer Vision, 3D Reconstruction