Go to NeurIPS 2023 Conference homepageITEM3D: Illumination-Aware Directional Texture Editing for 3D Models
Texture editing is a crucial task in 3D modeling that allows users to automatically manipulate the surface properties of 3D models. However, the inherent complexity of 3D models and the ambiguous text description lead to the challenge in this task. To address this challenge, we propose ITEM3D, an illumination-aware model for automatically 3D object editing according to the text prompts. Leveraging the power of diffusion model, ITEM3D takes the rendered images as the bridge of text and 3D representation and further optimizes the disentangled texture and environment map. Previous methods adopt the absolute editing direction namely score distillation sampling (SDS) as the optimization objective, which unfortunately results in the noisy appearance and text inconsistency. To solve the problem caused by the ambiguous text, we introduce a relative editing direction, a optimization objective defined by the noise difference between the source and target texts, to release the semantic ambiguity between the texts and images. Additionally, we gradually adjust the direction during optimization to further address the unexpected deviation in texture domain. Qualitative and quantitative experiments show that our ITEM3D outperforms SDS-based methods on various 3D objects. We also perform text-guided relighting to show the explicit control over lighting.