Learning Perceptual Compression of Facial Video

Mustafa Shukor, Xu YAO, Bharath Bhushan Damodaran, Pierre Hellier

Abstract: We propose in this paper a new paradigm for facial video compression. We leverage the generative capacity of GANs such as StyleGAN to represent and compress each video frame (intra compression), as well as the successive differences between frames (inter compression). Each frame is inverted in the latent space of StyleGAN, where the optimal compression is learned. To do so, a diffeomorphic latent representation is learned using a normalizing flows model, where an entropy model can be optimized for image coding. In addition, we propose a new perceptual loss that is more efficient than other counterparts (LPIPS, VGG16). Finally, an entropy model for inter coding with residual is also learned in the previously constructed latent space. Our method (SGANC) is simple, faster to train, and achieves competitive results for image and video coding compared to state-of-the-art codecs such as VTM, AV1, and recent deep learning techniques.