ToTem NRSfM: Object-wise Non-Rigid Structure-from-Motion with a Topological Template

Agniva Sengupta, Adrien Bartoli

Abstract:

We present a Non-Rigid Structure-from-Motion (NRSfM) method to reconstruct an object whose topology is
known. We represent the topology by a 3D shape that weakly resembles the object, which we call a Topological
Template (ToTem). The ToTem has two main differences with the template used in Shape-from-Template (SfT).
First, the shape in the ToTem is not necessarily feasible for the object, whereas it must be in the SfT template.
Second, the ToTem only models shape, excluding the classical texturemap representing colour in the SfT template.
These two differences greatly alleviate the practical difficulty of constructing a template. However, they make
the reconstruction problem challenging, as they preclude the use of strong deformation constraints between the
template shape and the reconstruction and the possibility of directly establishing correspondences between the
template and the images. Our method uses an isometric deformation prior and proceeds in four steps. First,
it reconstructs point clouds from the images. Second, it aligns the ToTem to the point clouds. Third, it creates a
coherent surface parameterisation. Fourth, it performs a global refinement, posed as a Bundle Adjustment (BA). We
show experimentally that our method outperforms the existing methods for its isolated steps and NRSfM methods
overall, in terms of 3D accuracy, ability to reconstruct the object’s visible surface and ability to approximate the
object’s invisible surface