NeRF-Enhanced Outpainting for Faithful Field-of-View Extrapolation

Rui Yu, Jiachen Liu, Zihan Zhou, Sharon X Huang

Abstract:

In various applications, such as robotic navigation
and remote visual assistance, expanding the field of view (FOV)
of the camera proves beneficial for enhancing environmental
perception. Unlike image outpainting techniques aimed solely
at generating aesthetically pleasing visuals, these applications
demand an extended view that faithfully represents the scene.
To achieve this, we formulate a new problem of faithful FOV
extrapolation that utilizes a set of pre-captured images as prior
knowledge of the scene. To address this problem, we present a
simple yet effective solution called NeRF-Enhanced Outpainting
(NEO) that uses extended-FOV images generated through NeRF
to train a scene-specific image outpainting model. To assess the
performance of NEO, we conduct comprehensive evaluations
on three photorealistic datasets and one real-world dataset.
Extensive experiments on the benchmark datasets showcase
the robustness and potential of our method in addressing this
challenge. We believe our work lays a strong foundation for
future exploration within the research community.