EX-NVS: EXtreme Novel View Synthesis via Depth Watertight Mesh

Tao Hu, Haoyang Peng, Chenguo Lin, Panwang Pan, Xiao Liu, Yuewen Ma

Keywords: Video Generation, Scene Generation, Camera Controllable, Mesh, Novel View Synthesis

TL;DR: We propose to achieve Extreme Novel View Synthesis via Depth Watertight Mesh

Abstract: We introduce EX-NVS, a framework that addresses these challenges via a Depth Watertight Mesh (DW-Mesh) representation that explicitly models both visible and occluded regions, providing a robust geometric prior across viewpoints. Unlike traditional surface reconstruction methods that struggle with sparse visibility, our DW-Mesh ensures complete geometric coverage and maintains watertight properties essential for extreme viewpoint synthesis. To overcome the requirement for multi-view paired training data, we propose a simulated masking strategy that produces effective supervision from common monocular videos. A lightweight LoRA-based video diffusion adapter with novel linear aggregation capabilities integrates the DW-Mesh priors to synthesize high-quality, physically consistent, and temporally coherent videos. Extensive experiments demonstrate that EX-NVS outperforms state-of-the-art methods across a variety of metrics, with particularly strong improvements for extreme camera angles ranging from -90° to 90°, enabling practical extreme novel view synthesis.

Primary Area: applications to computer vision, audio, language, and other modalities

Submission Number: 23793