Compressible-composable NeRF via Rank-residual Decomposition

Jiaxiang Tang, Xiaokang Chen, Jingbo Wang, Gang Zeng

Keywords: Neural Radiace Fields, Tensor Rank Decomposition, Compression, Composition.

Abstract:

Neural Radiance Field (NeRF) has emerged as a compelling method to represent 3D objects and scenes for photo-realistic rendering.
However, its implicit representation causes difficulty in manipulating the models like the explicit mesh representation.
Several recent advances in NeRF manipulation are usually restricted by a shared renderer network, or suffer from large model size.
To circumvent the hurdle, in this paper, we present a neural field representation that enables efficient and convenient manipulation of models.
To achieve this goal, we learn a hybrid tensor rank decomposition of the scene without neural networks.
Motivated by the low-rank approximation property of the SVD algorithm, we propose a rank-residual learning strategy to encourage the preservation of primary information in lower ranks.
The model size can then be dynamically adjusted by rank truncation to control the levels of detail, achieving near-optimal compression without extra optimization.
Furthermore, different models can be arbitrarily transformed and composed into one scene by concatenating along the rank dimension.
The growth of storage cost can also be mitigated by compressing the unimportant objects in the composed scene.
We demonstrate that our method is able to achieve comparable rendering quality to state-of-the-art methods, while enabling extra capability of compression and composition.
Code is available at https://github.com/ashawkey/CCNeRF.

TL;DR: We explore the compressibility and composability of NeRF in a tensor rank decomposition setting through rank-residual learning.

Community Implementations: [ 1 code implementation](https://www.catalyzex.com/paper/arxiv:2205.14870/code)