FaceZip: Automatic Texture Compression for Facial Blendshapes

Yiyi Cai, Shinichi Kinuwaki, Nobuyuki Umetani

Keywords: digital human, facial animation, blendshape, texture compression

TL;DR: We present a method to compress textures for facial blendshape targetting real-time applications.

Abstract: Recently, numerous cinematic and interactive entertainment production companies have adopted advanced capture systems to acquire faithful facial geometries and corresponding textures. However, it is difficult to animate these models in a controllable manner for real-time applications. Although blendshape is typically used for parameterizing facial geometries, dynamically changing the texture of the geometry is challenging. Because texture data are significantly larger than the vertex coordinates of the meshes, storing the texture of all the blendshape meshes is impractical. We present a method to compress texture data in a manner compatible with blendshape for real-time applications such as video games. Our method takes advantage of the locality of facial texture differences by blending a few textures with spatially different weights. Our method achieved a more accurate reconstruction of the original textures compared with the baseline principal component analysis.

Track: Graphics

Revision: No

Revision Reviewers: No opinion

Summary Of Changes:

Dear Primary:

We are very thankful for all the thoughtful comments and suggestions from the reviewers. In this revision, we managed to address all the issues raised in the review. All the major changes (excluding the small edits to fix typos) are highlighted in red. The major changes include:

• In Section 3.2, We explain the regularization we use in Equation 11
• In Section 4, We add more detail of our PCA implementation and how the RMSE loss is computed
• In Section 4, We further compare our result with PCA on different number of principle components (4 and 8) as the baseline. We show the result of PCA8 in green bars in Figure 6.
• In Section 4, We explain more about the choice of the number of difference textures
• In Section 4, We explain more about the data in Figure 7 (right)
• In Section 5, We discuss the factors that may affect the result of our methods and the future work
• We refine the expressions of the text using professional proof-reading service.

Again, we are very grateful for all the reviewers’ time and effort to improve the paper. We are happy to address any further issues.

Sincerely,
Authors