Spatially-aware Photo-realistic Face Relighting using Joint Embedding of Light Properties
Keywords: Face Relighting, Joint Light Property Embedding, Realistic Shadows
Abstract: Single image face relighting is the challenging problem of estimating the illumination cast on images by a point light source varying in position, intensity and possibly colour. Learning the relationship between the light source properties and the face location is critical to the photo-realism of the estimated relit image. Prior works do not explicitly model this relationship which adversely affects the accuracy and photo-realism of the estimated relit image. We present a novel framework that explicitly models this relationship by integrating a novel light feature embedding with self-attention and cross attention layers in a custom image relighting network. Our proposed method estimates more photo-realistic relit images with accurate shadows and outperforms prior works despite being trained only on synthetic data. Our method is able to generalize to out-of-training light source positions and also achieves unsupervised adaptation from synthetic to real images.
Supplementary Material: pdf
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 3841
Loading