Fast unsupervised consistent and modality-specific hashing for multimedia retrieval

Zhan Yang, Xiyin Deng, Jun Long

Abstract: Hashing is an effective technique to solve large-scale data storage problem and achieve efficient retrieval, and it is also a core technology to promote the intelligent development of the new infrastructure construction. In most practical situations, label information is unavailable, and creating manual annotations is a time-consuming and laborious process. Therefore, unsupervised cross-modal hashing technique has received extensive attention from the information retrieval community due to its fast retrieval speed and feasibility. However, the capabilities of existing unsupervised cross-modal hashing methods are not sufficient to comprehensively describe the complex relations among different modalities, such as the balance of complementary and consistency between different modalities. In this article, we propose a new-type of unsupervised cross-modal hashing method called Fast Unsupervised Consistent and Modality-Specific Hashing (FUCMSH). Specifically, FUCMSH consists of two main modules, i.e., shared matrix factorization module (SMFM) and individual auto-encoding module (IAEM). In the SMFM, FUCMSH dynamically assigns weights to different modalities to adaptively balance the contribution of different modalities. By doing so, the information completeness of the shared consistent representation can be guaranteed. In the IAEM, FUCMSH learns individual modality-specific latent representations of different modalities through modality-specific linear autoencoders. Moreover, FUCMSH makes use of the transfer learning to link the relationships between different individual modality-specific latent representations. Combined with the SMFM and the IAEM, the discriminative capability of the generated binary codes can be significantly improved. The relatively extensive experimental results manifest the superiority of the proposed FUCMSH.