Geometry-aware Score Distillation via 3D Consistent Noising and Gradients
Keywords: Diffusion Models, Score Distillation Sampling, Text-to-3D Generation
Abstract: Score distillation sampling (SDS), the methodology in which the score from pretrained 2D diffusion models is distilled into 3D representation, has recently brought significant advancements in text-to-3D generation. However, this approach is still confronted with critical geometric inconsistency problems such as the ``Janus problem''. We provide a novel insight into this problem, hypothesizing that the incorporation of 3D awareness into the 3D noising process and gradient distillation process may bring about enhanced consistency between gradients, leading to improved fidelity and geometric consistency. To achieve this, we propose a simple yet effective approach to achieve a 3D consistent, geometry-aware noising process, leveraging the advantages that 3D Gaussian Splatting possesses as an explicit 3D representation. Combined with our geometry-based gradient warping and our novel gradient dissimilarity loss, we demonstrate that our method significantly improves performance by addressing geometric inconsistency problems in text-to-3D generation with minimal computation cost and being compatible with existing score distillation-based models.
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5470
Loading