Reference-Guided Identity Preserving Face Restoration

Abstract: Preserving face identity is a critical yet persistent challenge in diffusion-based image restoration. While reference faces offer a path forward, existing methods typically suffer from partial reference information and inefficient identity losses. This paper introduces a novel approach that directly solves both issues, involving three key contributions: 1) Composite Context, a representation that fuses high- and low-level facial information to provide ensembled guidance than traditional singular representations, 2) Hard Example Identity Loss, a novel loss function that uses the reference face to address the identity learning inefficiencies of the standard identity loss, 3) Training-free multi-reference inference, a new method that leverages multiple references for restoration, despite being trained with only a single reference. The proposed method demonstrably restores high-quality faces and achieves state-of-the-art identity preserving restoration on benchmarks such as FFHQ-Ref and CelebA-Ref-Test, consistently outperforming previous work.

Submission Type: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

Compared to the previous version, we made changes in the following aspects:

* Clarity and Terminology: The introduction was clarified to explicitly state the main proposals—Composite Context and Hard Example Identity Loss—and the term "comprehensive representation" was replaced with "ensembled representation".
* Expanded Evaluation: Results for two previously omitted baselines, OSDFace and InstantRestore, were added to the appendix, along with a discussion that prioritizes identity preservation over general image quality.
* Structural Improvements: The appendix was proofread and reorganized with a table of contents, and a new section (Appendix A.2) was added to clearly summarize the method's contributions and differences from related works.

Assigned Action Editor: ~Søren_Hauberg1

Submission Number: 6792