NeRFlame: Flame-Based Conditioning of NeRF for 3D Face Rendering

Wojciech Zajac, Joanna Waczynska, Piotr Borycki, Jacek Tabor, Maciej Zieba, Przemyslaw Spurek

Abstract: Traditional 3D face models are based on mesh representations with texture. One of the most important models is Flame (Faces Learned with an Articulated Model and Expressions), which produces meshes of human faces that are fully controllable. Unfortunately, such models have problems with capturing geometric and appearance details. In contrast to mesh representation, the neural radiance field (NeRF) produces extremely sharp renders. However, implicit methods are hard to animate and do not generalize well to unseen expressions. It is not trivial to effectively control NeRF models to obtain face manipulation.