Dense Interspecies Face Embedding

Sejong Yang, Subin Jeon, Seonghyeon Nam, Seon Joo Kim

Keywords: Face Understanding, Cross Domain, Knowledge Distillation

TL;DR: We introduce a new task of cross-domain face understanding, and propose a dense interspecies face embedding (DIFE) learned in an unsupervised manner by our multi-teacher knowledge distillation and pseudo-paired data synthesis

Abstract: Dense Interspecies Face Embedding (DIFE) is a new direction for understanding faces of various animals by extracting common features among animal faces including human face. There are three main obstacles for interspecies face understanding: (1) lack of animal data compared to human, (2) ambiguous connection between faces of various animals, and (3) extreme shape and style variance. To cope with the lack of data, we utilize multi-teacher knowledge distillation of CSE and StyleGAN2 requiring no additional data or label. Then we synthesize pseudo pair images through the latent space exploration of StyleGAN2 to find implicit associations between different animal faces. Finally, we introduce the semantic matching loss to overcome the problem of extreme shape differences between species. To quantitatively evaluate our method over possible previous methodologies like unsupervised keypoint detection, we perform interspecies facial keypoint transfer on MAFL and AP-10K. Furthermore, the results of other applications like interspecies face image manipulation and dense keypoint transfer are provided. The code is available at https://github.com/kingsj0405/dife.