GenUDC: High Quality 3D Mesh Generation With Unsigned Dual Contouring Representation

Ruowei Wang, Jiaqi Li, Dan Zeng, Xueqi Ma, Xu Zixiang, Jianwei Zhang, Qijun Zhao

Abstract: Generating high-quality meshes with complex structures and realistic surfaces is the primary goal of 3D generative models. Existing methods typically employ sequence data or deformable tetrahedral grids for mesh generation. However, sequence-based methods have difficulty producing complex structures with many faces due to memory limits. The deformable tetrahedral grid-based method MeshDiffusion fails to recover realistic surfaces due to the inherent ambiguity in deformable grids. We propose the novel GenUDC framework to address these challenges, leveraging the Unsigned Dual Contouring (UDC) as a better mesh representation. UDC discretizes a mesh in a regular grid and divides it into the face and vertex parts, recovering both complex structures and fine details. As a result, the one-to-one mapping between UDC and mesh resolves the ambiguity problem. In addition, GenUDC adopts a two-stage, coarse-to-fine generative process for 3D mesh generation. It first generates the face part as a rough shape and then the vertex part to craft a detailed shape. Extensive evaluations demonstrate the superiority of UDC as a mesh representation and the favorable performance of GenUDC in mesh generation. The code and trained models will be released upon publication.

Primary Subject Area: [Generation] Generative Multimedia

Secondary Subject Area: [Experience] Multimedia Applications

Relevance To Conference:

Our research aligns well with the multimedia theme, especially in the context of "Multimedia in the Generative AI Era" and "Multimedia Applications". By leveraging advanced generative models, including diffusion models, our research contributes significantly to the field of multimedia by enabling the generation of 3D mesh with unprecedented realism and variety. By integrating our 3D mesh generation techniques, multimedia applications can offer more immersive experiences, from virtual reality environments to interactive educational tools, thus significantly contributing to the theme.

We recognize that this conference has published many papers on similar research topics, such as text-to-3D, 3D scene generation, 3D point cloud generation, 3D avatar generation, image-to-3D, 3D reconstruction, and 3D human reconstruction. Our method proposes a new paradigm for 3D generation and 3D shape modeling, which can be the foundation of various 3D tasks, such as text-to-3D, image-to-3D, 3D completion, 3D reconstruction, 3D shape retrieval, and so on.

Submission Number: 369