High-Quality Geometry and Texture Editing of Neural Radiance Field

Joo Ho Lee

Abstract:

Recent advances in Neural Radiance Field (NeRF) have demonstrated impressive rendering quality reconstructed from input
images. However, the density-based radiance field representation introduces entanglement of geometry and texture, limiting the
editability. To address this issue, NeuMesh proposed a mesh-based NeRF editing method supporting deformation and texture
editing. Still, it fails reconstructing and rendering fine details of input images, and the dependency between rendering scheme
and geometry limits editability for target scenes. In this paper, we propose an intermediate scene representation where a
near-surface volume is associated with the guide mesh. Our key idea is separating a given scene into geometry, parameterized
texture space, and radiance field. We define a mapping between xyz-coordinate space and uvh-coordinate system defined by
combination of mesh parameterization and the height from mesh surface to efficiently encode the near-surface volume. With
the surface-aligned radiance field defined in the near-surface volume, our method can generate high quality rendering results
with high frequency details. Our method also supports various geometry and appearance editing operations while preserving
high rendering quality. We demonstrate the performance of our method by comparing it with the state-of-the-art methods both
qualitatively and quantitatively and show its applications including shape deformation, texture filling, and texture painting.