ART-FR: masked Auto-Regressive Transformer for Face Restoration
Keywords: Auto-regressive model; Face restoration
Abstract: Restoring authentic facial features from low-quality images presents an extremely challenging task, due to the intricate real-world degradations and the inherently ill-posed nature of the problem. Existing methods, which utilize a codebook prior, help alleviate the complexity of the restoration process and produce visually plausible outcomes. However, these methods struggle to accurately capture the mapping between low-quality (LQ) and high-quality (HQ) images in the discrete latent space, leading to suboptimal results.
Inspired by the success of auto-regressive generation paradigm in discrete modeling problems (e.g. large language models), we propose an Auto-Regressive Transformer based Face Restoration (ART-FR) method to mitigate this mapping challenge. Specifically, with the aid of a visual tokenizer, we reformulate the face restoration task as a conditional generation problem within the discrete latent space. Furthermore, a masked generative image transformer is employed to model the distribution of this latent space, conditioned on LQ features. Face restoration is subsequently performed in the latent space through iterative sampling, with the HQ image reconstructed using a pretrained decoder. Extensive experimental validation demonstrates ART-FR exhibits superior performance across various benchmark datasets.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 4486
Loading