A Multiple-View Geometric Model of Specularities on Non-Planar Shapes with Application to Dynamic Retexturing

Alexandre Morgand, mohamed Tamaazousti, Adrien Bartoli

Abstract:

Predicting specularities in images, given the camera pose and scene geometry from SLAM, forms a challenging and open
problem. It is nonetheless essential in several applications such as retexturing. A recent geometric model called JOLIMAS partially
answers this problem, under the assumptions that the specularities are elliptical and the scene is planar. JOLIMAS models a moving
specularity as the image of a fixed 3D quadric. We propose dual JOLIMAS, a new model which raises the planarity assumption. It uses
the fact that specularities remain elliptical on convex surfaces and that every surface can be divided in convex parts. The geometry of
dual JOLIMAS then uses a 3D quadric per convex surface part and light source, and predicts the specularities by a means of virtual
cameras, allowing it to cope with surface’s unflatness. We assessed the efficiency and precision of dual JOLIMAS on multiple synthetic
and real videos with various objects and lighting conditions. We give results of a retexturing application. Further results are presented
as supplementary video material.