MVPaint: 3D Texture Generation with Multi-View Consistency

Wei Cheng, Juncheng Mu, Xianfang Zeng, Xin Chen, Anqi Pang, Chi Zhang, Zhibin Wang, Bin Fu, Gang Yu, Ziwei Liu, Liang Pan

14 Sept 2024 (modified: 04 Oct 2024)ICLR 2025 Conference Withdrawn SubmissionEveryoneRevisionsBibTeXCC BY 4.0
Keywords: texture generation
Abstract: Texturing is a crucial step in the 3D asset production workflow, which enhances the visual appeal and diversity of 3D assets. Despite recent advancements in generation-based texturing, existing methods often yield subpar results, primar- ily due to local discontinuities, inconsistencies across multiple views, and their heavy dependence on UV unwrapping outcomes. To tackle these challenges, we propose a novel generation-refinement 3D texturing framework called MV- Paint, which can generate high-resolution, seamless textures while emphasizing multi-view consistency. Given a 3D mesh model, MVPaint first simultaneously generates multi-view images by employing a Synchronized Multi-view Genera- tion (SMG) module, which leads to coarse texturing results with unpainted parts due to missing observations. To ensure complete 3D texturing, we introduce the Spatial-aware 3D Inpainting (S3I) method, specifically designed to texture pre- viously unobserved areas effectively. Furthermore, MVPaint employs a UV Re- finement (UVR) module for improving the texture quality in the UV space. UVR first performs a UV-space Super-Resolution, followed by a Spatial-aware Seam- Smoothing algorithm for revising spatial texturing discontinuities caused by UV unwrapping. Moreover, we perform meticulous manual annotations to filter the Objaverse dataset, resulting in around 100,000 high-quality 3D data for texturing generation. Extensive experimental results demonstrate that MVPaint surpasses existing state-of-the-art methods. Notably, MVPaint could generate high-fidelity textures with fewer multi-face issues and better cross-view consistency
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Resubmission: No
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 729