StyleDreamer: Make Your 3D Style Avatar from a Single View with Consistency Score Distillation

Zhenglin Zhou, Fan Ma, Hehe Fan, Yi Yang

Keywords: 3D generation, Stylized 3D head avatar, Single view

TL;DR: This paper propose to use one view to make a 3D style avatar.

Abstract:

Recent generative methods have shown remarkable capabilities in producing stylized 3D head avatars. Nonetheless, current methods necessitate multi-view images for constructing 3D models, which limits their usage in the real world where most images are captured from a single angle. In this paper, we investigate a practical $\textit{One-to-Style}$ task, generating the 3D style avatar with a single view.
The task presents two challenges: 1) $\textbf{Content}$ consistency and 2) $\textbf{Style}$ consistency across multiple views of the generated images. We propose a novel Consistency Score Distillation (CSD) to ensure consistent stylization across multiple views while preserving the identity of each view using the 3D GAN. In this method, the style distribution of the rendered images from all views is supervised to match the style of the given single view, based on the provided edit instruction. We have formulated a dataset for One-to-Style with in-the-wild face images and the most commonly used style. Experimental results show that our approach outperforms existing methods in terms of stability and quality, indicating its potential applications in the real world. Result videos can be found on the project website: https://one-to-style.github.io/.

Primary Area: generative models

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 7456