A Framework for Single-View Multi-Plane Image Inpainting

Zachary McBride Lazri, Dae Yeol Lee, Guan-Ming Su

Abstract: The multi-plane image (MPI) is a promising framework for 3D scene reconstruction. However, efficient bitstream transmission of multiple MPIs may be infeasible for most current market products. While an MPI from a single camera position can be used to generate local novel views of a scene, errors produced in the MPI generation process may create artifacts in the rendered images produced at novel views. Thus, we introduce a novel framework for MPI inpainting, which restores the quality of a single MPI that is used for novel view synthesis. We assume that multi-view MPIs from different camera positions are taken of a scene, but only one is capable of being transmitted over a bitstream. We leverage the pixel information contained in MPIs captured at local neighboring views to inpaint a single MPI by: (1) detecting the location of the artifacts produced by an MPI at a novel view and (2) using neighboring views' pixels to fill the MPI at its original location so that when warped to novel views the artifacts are removed. Experimental results on three datasets verify that our inpainting framework can enable high-quality rendering at local novel views with significant artifact reduction.

External IDs:dblp:conf/mipr/LazriLS24