CoDi: Contrastive Disentanglement Generative Adversarial Networks for Zero-Shot Sketch-Based 3D Shape Retrieval

Min Meng, Wenhang Chen, Jigang Liu, Jun Yu, Jigang Wu

Abstract: Sketch-based 3D shape retrieval has attracted increasing attention in recent years. Most existing methods fail to address the zero-shot scenario, and the few dedicated to zero-shot learning encounter the following two issues: 1) the features learned by these methods lack informativeness and generalization, rendering them ineffective in identifying unseen samples; 2) the generation of low-quality samples, aimed at facilitating the recognition of unseen categories, paradoxically diminishes their ability to identify these unseen classes. This paper introduces a novel contrastive disentanglement generative adversarial networks (CoDi) tailored for zero-shot sketch-based 3D shape retrieval. Initially, we introduce a paradoxical feature construction approach designed to assist the networks in capturing certain low-level features. Despite their weak semantic relevance, these features play a crucial role in sample recognition. Subsequently, a SemContrast fusion module is employed to align the semantic space with the prototype embedding space of categories. This alignment facilitates knowledge transfer to unseen classes and promotes the generation of high-quality samples. The networks are jointly trained on real and generated samples to achieve retrieval for unseen categories. Extensive experiments demonstrate a significant improvement in retrieval performance for unseen categories using our method.