GVP: Generative Volumetric Primitives

MALLIKARJUN BYRASANDRA RAMALINGA REDDY, Xingang Pan, Mohamed Elgharib, Christian Theobalt

Keywords: 3D Generative Model, Face Modelling, Morphable Models

Abstract:

Advances in 3D-aware generative models have pushed the boundary of image synthesis with explicit camera control.
To achieve high-resolution image synthesis, several attempts have been made to design efficient generators, such as hybrid architectures with both 3D and 2D components.
However, such a design compromises multiview consistency, and the design of a pure 3D generator with high resolution is still an open problem.
In this work, we present Generative Volumetric Primitives (GVP), the first pure 3D volumetric generative model that can sample and render 512-resolution images in real-time.
GVP jointly models a number of volumetric primitives and their spatial information, both of which can be efficiently generated via a 2D convolutional network.
The mixture of these primitives naturally captures the sparsity in the 3D volume.
The training of such a generator with a high degree of freedom is made possible through a combination of adversarial and knowledge distillation training.
The learned model exhibits dense 3D correspondences between samples. We provide exhaustive qualitative and qualitative evaluations for dense correspondences.
Experiments on several datasets demonstrate superior efficiency, 3D consistency, and the emergence of dense correspondences of GVP over the state-of-the-art.

Submission Number: 248