HierT2S: Enhancing Part-Level Text-to-Shape Generation via Hierarchical Structure Modeling
Keywords: 3D shape generation, text-to-shape
Abstract: Text-driven 3D shape generation still faces key challenges, especially in achieving high levels of control over the generated outputs. Paticularly, existing text-to-shape methods ignore the explicit modeling of hierarchical structures in the text and 3D shapes, which makes it hard for using long text descriptions with multiple prompts to guide the coherent part-level 3D shape generation. In this work, we introduce HierT2S, a framework that integrates a hierarchical tree representation with a conditional diffusion model, to enhance the generation of 3D shapes with coherent structures induced by the hierarchical and structured text representations. The key idea is to first segment the input text into several clusters and construct a hierarchical tree representation, with each node representing a parent entity or the fine-level part components. Then, we process the lower-level clusters of the tree with a relation graph module which uses self-attention mechanism to aggregate the relationships of the clusters, and generate a new sequence containing the processed text features. Finally, the text features are embedded into the 3D feature space and used for learning the 3D shape generation by a conditional diffusion model, where the sparsely implicit parsed hierarchical tree graph further enhances the structural details of the generated 3D shapes, leading to results that are close to structure-aware generation. We conducted comprehensive experiments on the existing text-to-shape pairing dataset Text2Shape, and the results demonstrate that our model significantly outperforms current state-of-the-art methods. Moreover, our method can enable progressive part-level 3D shape manipulation and modification guided by the partially modified text prompt.
Primary Area: generative models
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 7727
Loading