Generating CAD Code with Vision-Language Models for 3D Designs
Keywords: Code Generation, Self-refinement
TL;DR: we develop a Self-Verification approach, wherein LLMs generate 3D objects through scripting code, and then verify whether the generated 3D objects meet the requirements.
Abstract: Generative AI has transformed the fields of Design and Manufacturing by providing
efficient and automated methods for generating and modifying 3D objects. One
approach involves using Large Language Models (LLMs) to generate Computer-
Aided Design (CAD) scripting code, which can then be executed to render a 3D
object; however, the resulting 3D object may not meet the specified requirements.
Testing the correctness of CAD generated code is challenging due to the complexity
and structure of 3D objects (e.g., shapes, surfaces, and dimensions) that are not
feasible in code. In this paper, we introduce CADCodeVerify, a novel approach to
iteratively verify and improve 3D objects generated from CAD code. Our approach
works by producing ameliorative feedback by prompting a Vision-Language Model
(VLM) to generate and answer a set of validation questions to verify the generated
object and prompt the VLM to correct deviations. To evaluate CADCodeVerify, we
introduce, CADPrompt, the first benchmark for CAD code generation, consisting of
200 natural language prompts paired with expert-annotated scripting code for 3D
objects to benchmark progress. Our findings show that CADCodeVerify improves
VLM performance by providing visual feedback, enhancing the structure of the 3D
objects, and increasing the success rate of the compiled program. When applied to
GPT-4, CADCodeVerify achieved a 7.30% reduction in Point Cloud distance and a
5.0% improvement in success rate compared to prior work.
Supplementary Material: zip
Primary Area: applications to computer vision, audio, language, and other modalities
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 13116
Loading