Blind Face Video Restoration with Temporal Consistent Generative Prior and Degradation-Aware Prompt

Jingfan Tan, Hyunhee Park, Ying Zhang, Tao Wang, Kaihao Zhang, Xiangyu Kong, Pengwen Dai, Zikun Liu, Wenhan Luo

Abstract: Within the domain of blind face restoration (BFR), approaches lacking facial priors frequently result in excessively smoothed visual outputs.Exiting BFR methods predominantly utilize generative facial priors to achieve realistic and authentic details. However, these methods, primarily designed for images, encounter challenges in maintaining temporal consistency when applied to face video restoration. To tackle this issue, we introduce StableBFVR, an innovative Blind Face Video Restoration method based on Stable Diffusion that incorporates temporal information into the generative prior. This is achieved through the introduction of temporal layers in the diffusion process.These temporal layers consider both long-term and short-term information aggregation.Moreover, to improve generalizability, BFR methods employ complex, large-scale degradation during training, but it often sacrifices accuracy. Addressing this, StableBFVR features a novel mixed-degradation-aware prompt module, capable of encoding specific degradation information to dynamically steer the restoration process.Comprehensive experiments demonstrate that our proposed StableBFVR outperforms state-of-the-art methods.

Primary Subject Area: [Content] Vision and Language

Relevance To Conference: This work is an blind face video restoration approach. It leverages the generative prior from the pre-trained Stable Diffusion to restore face videos with realistic details. And we introduce temporal layers to ensure content consistency among frames.

Submission Number: 908