Magic Tailor: a Latent-Anchor based Diffusion Model for 3D Clothes Generation

Zibo Zhao, Wen Liu, Xin Chen, Gang YU, Shenghua Gao

Abstract: We study conditional 3D clothes generation to synthesize high-quality 3D clothes models that conform to various conditions, such as clothes categories, images, and texts. Traditional methods to generate 3D clothes depend on registering 3D clothes to human parametric models or predefined templates. However, this registration process inevitably compromises the fidelity and topology of clothes. Thus, we propose a topology-free and computation-friendly latent-anchor representation for 3D clothes to tackle this restriction. Specifically, we employ a Vector QuantisedVariational AutoEncoder (VQ-VAE) to encode each 3D clothes model into groups of latent anchors, and each latent anchor contains an anchor point and anchor embedding. Based on the latent-anchor representation, we introduce a novel two-level latent-anchor diffusion model (LAD) that first learns a probabilistic mapping function from various conditional inputs to anchor points. The anchor points and conditional inputs are used to generate the anchor embeddings. Then, anchor points and anchor embeddings are fed into the decoder of VQ-VAE for 3D clothes generation. Extensive experimental results demonstrate the effectiveness of LAD in producing 3D clothes models. The codes of our work will be released later to facilitate further research in this field.