Cbir-Gan: A Triplet Generative Adversarial Network for Content-Based Image Retrieval

Narges Norouzi, Maral Zarvani

Abstract: Content-based image retrieval systems have become popular in various fields such as computer vision and artificial intelligence. Deep neural networks, especially CNNs, have frequently been employed in visual representations; however, they require large amounts of labeled data, which are hard, costly, and sometimes impossible to obtain. Moreover, since these methods rely only on semantically discriminative representations, they fail to yield significant outputs in instancelevel image retrieval systems. Therefore, this paper proposes a triplet generative adversarial network (GAN) based on the idea of integrating deep metric learning methods with GANs to benefit from the advantages of both at the same time. In this model, three generator networks that use a triplet loss function are responsible for learning a similarity measure over objects and embedding images in an appropriate vector space. In these networks, a CNN-based perceptual loss function is also employed to force the generators to adhere a certain type of structural features in intermediate layers. Since only triplets must be used as network inputs in the proposed method, the learning process is performed in a semi-supervised way. According to the results of comprehensive experiments conducted on four datasets in comparison with several state-of-the-art methods, the proposed method was efficient in terms of precision and computational complexity. For the real-life implementation of the proposed method, a distributed large-scale fashion image retrieval platform, called SnapMode1, has been developed through big data tools such as Apache Storm, Kafka, Solr, and Milvus.