Learning Compact Semantic Information for Incomplete Multi-View Missing Multi-Label Classification

Jie Wen, Yadong Liu, Zhanyan Tang, Yuting He, Yulong Chen, Mu Li, Chengliang Liu

TL;DR: We propose a novel cross-view shared information learning framework for incomplete multi-view missing multi-label classification.

Abstract: Multi-view data involves various data forms, such as multi-feature, multi-sequence and multimodal data, providing rich semantic information for downstream tasks. The inherent challenge of incomplete multi-view missing multi-label learning lies in how to effectively utilize limited supervision and insufficient data to learn discriminative representation. Starting from the sufficiency of multi-view shared information for downstream tasks, we argue that the existing contrastive learning paradigms on missing multi-view data show limited consistency representation learning ability, leading to the bottleneck in extracting multi-view shared information. In response, we propose to minimize task-independent redundant information by pursuing the maximization of cross-view mutual information. Additionally, to alleviate the hindrance caused by missing labels, we develop a dual-branch soft pseudo-label cross-imputation strategy to improve classification performance. Extensive experiments on multiple benchmarks validate our advantages and demonstrate strong compatibility with both missing and complete data.

Lay Summary: For the task of incomplete Multi-view Multi-label Classification (iM3C), we propose a strategy to extract multi-view shared semantic information relevant to downstream tasks from multiple views while discarding redundant information. Furthermore, we employ a dual-branch architecture to generate soft pseudo-labels for alternating training, which assists the model in achieving good performance even in the presence of missing labels. Experimental results on multiple datasets demonstrate the effectiveness of the proposed method.

Primary Area: General Machine Learning->Supervised Learning

Keywords: Multi-view multi-label classification, Incomplete multi-view learning, Missing multi-label classification

Submission Number: 6513