Lightplane: Highly-Scalable Components for Neural 3D Fields

Ang Cao, Justin Johnson, Andrea Vedaldi, David Novotny

Keywords: Volumetric Rendering, 3D Generation, 3D Reconstruction

Abstract:

Contemporary 3D research, particularly in reconstruction and generation, heavily relies on 2D images for inputs or supervision.
However, current designs for these 2D-3D mapping are memory-intensive, posing a significant bottleneck for existing methods and hindering new applications.
In response, we propose a pair of highly scalable components for 3D neural fields: Lightplane Renderer and Splatter, which significantly reduce memory usage in 2D-3D mapping (over $\bf{1000\times}$).
These innovations enable the processing of vastly more and higher resolution images with significantly small memory and computational costs.
We demonstrate their utility across various applications, from optimizing with image-level losses to enabling a versatile pipeline for scaling 3D reconstruction and generation.

Submission Number: 170