Computer-Aided Design as Language

Yaroslav Ganin, Sergey Bartunov, Yujia Li, Ethan Anderson Keller, Stefano Saliceti

Keywords: generative models, structured objects, computer-aided design, transformers, pointers, program synthesis

Abstract: Computer-Aided Design (CAD) applications are used in manufacturing to model everything from coffee mugs to sports cars. These programs are complex and require years of training and experience to master. A component of all CAD models particularly difficult to make are the highly structured 2D sketches that lie at the heart of every 3D construction. In this work, we propose a machine learning model capable of automatically generating such sketches. Through this, we pave the way for developing intelligent tools that would help engineers create better designs with less effort. The core of our method is a combination of a general-purpose language modeling technique alongside an off-the-shelf data serialization protocol. Additionally, we explore several extensions allowing us to gain finer control over the generation process. We show that our approach has enough flexibility to accommodate the complexity of the domain and performs well for both unconditional synthesis and image-to-sketch translation.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.

TL;DR: A method for generating structured objects (e.g., CAD sketches) by applying language modeling to an off-the-shelf serialization protocol.

Code: https://github.com/deepmind/deepmind-research/tree/master/cadl