FullFormer: Generating Shapes Inside Shapes

Tejaswini Medi, Jawad Tayyub, Muhammad Sarmad, Frank Lindseth, Margret Keuper

Abstract: Implicit generative models have gained significant popularity in the realm of 3D data modeling and have recently demonstrated their efficacy in both encoding and producing high-quality 3D shapes. However, prevailing research predominantly focuses on generating outer shells of 3D shapes, ignoring internal geometric details. In this work, we alleviate this limitation by presenting an implicit generative model that facilitates the generation of complex 3D shapes with rich internal structures. Our proposed model utilizes unsigned distance fields, enabling the representation of nested 3D shapes by learning from watertight and non-watertight data. Furthermore, We employ a transformer-based auto-regressive model for shape generation that leverages context-rich tokens from vector quantized shape embeddings. The generated tokens are decoded into unsigned distance field values which further render into novel 3D shapes exhibiting intrinsic details. We demonstrate that our model achieves state-of-the-art point cloud generation results on the popular ShapeNet classes ’Cars’, ’Planes’, and ’Chairs’. Further, we curate a dataset that exclusively comprises of shapes with realistic internal details from the ‘Cars’ category of ShapeNet, denoted FullCars. This dataset allows us to demonstrate our method’s efficacy in generating shapes with rich internal geometry. The code is available at FullFormer.