Restore3D: Breathing Life into Broken Objects with Shape and Texture Restoration

Xiaolong Shen, Zongxin Yang, Yi Yang

Keywords: Diffusion Models, 3D object completion, Multi-view Image Generation, Multi-view Image Inpainting

Abstract:

Restoring incomplete or damaged 3D objects is crucial for cultural heritage preservation, occluded object reconstruction, and artistic design.
Existing methods primarily focus on geometric completion, often neglecting texture restoration and struggling with relatively complex and diverse objects.
We introduce Restore3D, a novel framework that simultaneously restores both the shape and texture of broken objects using multi-view images. To address limited training data, we develop an automated data generation pipeline that synthesizes paired incomplete-complete samples from large-scale 3D datasets.
Central to Restore3D is a multi-view model, enhanced by a carefully designed Mask Self-Perceiver module with a Depth-Aware Mask Rectifier.
The rectified masks, learned through the self-perceiver, facilitate an image integration and enhancement phase that preserves shape and texture patterns of incomplete objects and mitigates the low-resolution limitations of the base model, yielding high-resolution, semantically coherent, and view-consistent multi-view images.
A coarse-to-fine reconstruction strategy is then employed to recover detailed textured 3D meshes from refined multi-view images. Comprehensive experiments show that Restore3D produces visually and geometrically faithful 3D textured meshes, outperforming existing methods and paving the way for more robust 3D object restoration. Project page: https://nip-ss.github.io/NIPS-anonymous/ .

Primary Area: Applications (e.g., vision, language, speech and audio, Creative AI)

Submission Number: 13842