Addressing Low-Shot MVS by Detecting and Completing Planar Surfaces

Rajbir Kataria, Zhizhong Li, Joseph DeGol, Derek Hoiem

Abstract: Multiview stereo (MVS) systems typically require at least three views to reconstruct each scene point. This requirement increases the burden of image captures and leads to incomplete reconstructions. Our main idea to address this low-shot MVS problem is to detect planar surfaces in depth maps generated by any MVS system and complete these surfaces by reformulating the MVS depth prediction task to a simpler planar surface assignment problem. We use single and multi-view cues (when available) and employ the DeepLabv3 architecture to infer the extent of planar regions and accurately complete missing surfaces. We show that our approach reconstructs portions of surfaces viewed by only one image, yielding denser models than existing MVS systems.