Neural implicit mapping via nested neighborhoods: real-time rendering of neural SDFs with textures

Vinícius Da Silva, Tiago Novello, Guilherme Schardong, Luiz Schirmer, HELIO CORTES VIEIRA LOPES, Luiz Velho

Primary Area: representation learning for computer vision, audio, language, and other modalities

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Keywords: Implicit Functions; Neural Signed Distance Functions; Real-time inference;

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

TL;DR: The nested neighborhood model addresses the open problem of real-time joint estimation of surface geometry, and attributes (normals, and textures) from dynamic neural SDFs.

Abstract:

We introduce the nested neighborhood model, a framework to address the problem of real-time joint estimation of surface geometry and its attributes (normals and textures) from neural SDFs.
This problem was only partially approached by previous works, which do not support attributes nor dynamic surfaces in real-time.
The framework is built on the nesting condition, which establishes a criteria for the neighborhoods of zero-level sets of a sequence of neural SDFs to be nested. This allows mappings between such neighborhoods, enabling the definition of the multiscale sphere tracing, the neural attribute mapping, and the GEMM-based analytical normal computation algorithms, composing the nested neighborhood model.
Our framework does not use spatial data-structures and its components can be used to augment meshes with smooth neural normals and textures. The normal GEMM-based computation does not depend on auto-differentiation nor computational graphs, resulting in real-time performance.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4272