Learning Detailed 3D Face via CLIP Model from Monocular Image

Pengfei Zhou, Yongtang Bao, Yue Qi

Abstract:

3D morphable face models (3DMMs) methods cannot accurately estimate facial expressions and geometric details. We propose a
framework for regressing 3D facial expressions and geometric details to address this problem. First, we propose a parameter re-
finement module to learn rich feature representations. Second, a novel feature consistency loss during training is designed, which
exploits the powerful representation ability of CLIP (Contrastive Language-Image-Pretraining) to capture facial expressions and geometric details. Finally, we leverage text-guided expression-specific transfer for 3D face reconstruction. Our method achieves significant performance in terms of reconstructed expressions and geometric details.