Autoencoder and Classifier based Joint-Guided Completion for Partial Multi-Modal Hashing

Xu Yi, Anchi Chen

Keywords: partial multi-modal hashing, autoencoder, classifier, Joint-Guided Completion

TL;DR: We propose a method named JCPMH to solve the partial multi-modal hashing problem by completing the missing modal data.

Abstract:

The Multi-Modal Hashing (MMH) method based on complete modalities cannot effectively handle incomplete multi-modal samples, thus requiring the completion of missing modalities. Existing completion methods typically use complete modality samples with the same label to generate completion information. On one hand, they cannot fully utilize the different information between samples with different labels; on the other hand, they cannot effectively extract the global structural information of multi-modal samples. Therefore, we propose the autoencoder and classifier based joint-guided completion for partial multi-modal hashing (JCPMH) method that integrates autoencoders and classifiers. First, to fully utilize the different information between samples with different labels, we design a multi-modal classification module composed of multiple classifiers to learn different information. Second, we concatenate the multi-modal data into a whole and extract cross-modal global structural information through an autoencoder. Finally, based on the hashing module, multi-modal classification module and autoencoder module, we design a loss function to guide the generator to generate more accurate completion information for learning hash codes. JCPMH can utilize partial multi-modal samples for offline training and handle incomplete multi-modal samples during online retrieval. Additionally, we conducted extensive experiments to demonstrate the effectiveness of this model.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 4097