Multimodal Nonredundant Clustering via Sufficiency Complementary Mining

Yiqiao Mao, Xiaoqiang Yan, Qiang Guo, Yangdong Ye

Abstract: The development of industrial Internet of Things (IoT) generates massive unlabeled multimodal data. multimodal clustering (MMC) methods can unsupervisedly learn reasonable clustering structures by mining the complementary information in each modality, which has become one of the essential critical technologies in industrial IoT. However, current MMC methods ignore redundant information in the multimodal data. This indistinguishable redundant information hinders the mining of complementary information and thus reduces the performance of MMC. In this article, we propose a multimodal nonredundant clustering via sufficiency complementary mining that explores complementary information while maximizing the removal of redundant information. We first construct a multimodal autoencoding network to learn the single-modal and fusion representation of all modalities. Then, we design a sufficiency complementary mining (SCM) mechanism based on information theory to train this autoencoding network. SCM transforms the complementary information into the sufficiency of fusion representation for each modality while reconstructing the redundant information as a conditional mutual information between the fusion and each single-modal representation given original data. This ingenious design allows SCM to safely remove redundant information by minimizing this conditional mutual information under guaranteed sufficiency. Moreover, the clustering task and the multimodal autoencoding network are integrated into a unified framework to enable the learned complementary information to guide the clustering process. Extensive experimental results on eight large-scale real-world datasets show that our method outperforms the state-of-the-art MMC methods.