Incomplete Multi-Modal Weakly-Supervised Hashing With Consensus Bipartite Graph

Xu Lu, Liang Zhang, Li Liu, Huaxiang Zhang

Abstract: Due to its excellent query and storage efficiency to facilitate large-scale multimedia retrieval, multi-modal hashing (MMH) has garnered a lot of attention from researchers. Nevertheless, existing MMH methods still suffer from several challenges: 1) Existing MMH methods often rely on graphs to represent complex correlation, but are constrained by the quality of graph construction and the storage overhead. 2) Existing MMH methods only deal with complete multi-modal data where all modalities of each instance are available, but cannot work with incomplete multi-modal data which encounter the problem of missing modalities. 3) Existing MMH methods often ignore the inevitable weak-supervision issue. To address these challenges, this paper proposes an Incomplete Multi-modal wEakly-supervised Hashing with Consensus Bipartite Graph (IMEH-CBG) method, which learns consensus bipartite graph for incomplete multi-modal fusion and corrects weak labels for discriminant hash learning. As far as we know, this is the first MMH method to work with incomplete and weakly-supervised multi-modal data in an unified framework. IMEH-CBG selects unified anchor set and builds consensus bipartite graph jointly for incomplete multi-modal fusion to tackle the first and the second challenges. Then, the semantic labels are predicted and utilized to learn hash code in an asymmetric way to tackle the third challenge. Extensive experiments demonstrate the superiority of IMEH-CBG.

External IDs:dblp:journals/tcsv/LuZLZ25