Light-Implicit Uncalibrated Photometric Stereo Network With Fourier Embedding
Primary Area: representation learning for computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Photometric Stereo, 3D Reconstruction, Fourier Transform, Amplitude and Phase, Uncalibrated light
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: We present a one-stage deep uncalibrated photometric stereo (UPS) network for non-Lambertian objects. Previous two-stage deep UPS networks estimated surface normals based on learned lighting because lighting is tangled with shading cues, making it challenging to directly estimate surface normals. However, two-stage UPS networks face fewer interpretations with embedded light direction's role in decomposing shading cues. Additionally, these two-stage methods discretize the light direction estimations instead of regressing exact light directions due to the learning difficulty and instability. However, the inexact light directions mislead shading cues extracted by the normal estimation network.
In contrast to previous two-stage UPS methods, our UPS-FourNet implicitly learns lighting by decomposing inputs using embedded Fourier transform. Our approach is motivated by a unique observation from photometric stereo images in the Fourier domain: lighting information predominantly concentrates on phases while shape information is closely related to amplitudes. By leveraging this property, the shape and lighting can be "decomposed" to a certain extent in the Fourier domain, eliminating the need for explicitly learning light directions and using them in the subsequent normal regression network. UPS-FourNet relaxes the limitations of two-stage UPS methods, with better training stability, concise end-to-end structures, and avoiding the discrete classification errors of light directions. Experiments on synthetic and real datasets show that our method achieves competitive results, which may push a new strategy for deep learning-based UPS methods.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1169
Loading