Heterogeneity-Aware Federated Deep Multi-View Clustering towards Diverse Feature Representations

Xiaorui Jiang, Zhongyi Ma, Yulin Fu, Yong Liao, Pengyuan Zhou

Abstract: Multi-view clustering has garnered increasing attention in recent years because of its ability to extract consistent and complementary information from multi-view data. In this context, contrastive learning has often been employed to explore the common semantics across different views. However, we notice that existing multi-view contrastive clustering methods often overlook cases where samples belonging to the same cluster but different views are incorrectly classified as negative feature pairs, leading to larger separations between features belonging to the same cluster in the feature space. To address this issue, we propose to shift the perspective from the view-level to the cluster-level and introduce $\mathbf{C}$luster-level $\mathbf{C}$ontrastive $\mathbf{D}$eep $\mathbf{M}$ulti-$\mathbf{V}$iew $\mathbf{C}$lustering ($\mathbf{CCDMVC}$) method based on an intra-cluster negative pair exemption strategy. Specifically, by constructing global features to utilize complete view information, we infer the clustering probability of each sample, thus reducing the construction of negative feature pairs belonging to the same cluster. As a result, the contrastive loss is corrected, allowing the model to treat different levels of feature pairs differently, minimizing the introduction of noise and making the sample points within the same cluster more compact. Additionally, we propose a cluster-level imputation module to make CCDMVC compatible with scenarios involving incomplete data. This module infers missing features with high confidence clustering probabilities and classifies them in cluster-level form. We conduct extensive experiments on eight datasets with fourteen baseline algorithms. The results demonstrate that CCDMVC exhibits superior clustering performance.

Primary Subject Area: [Content] Multimodal Fusion

Secondary Subject Area: [Content] Multimodal Fusion, [Systems] Systems and Middleware, [Experience] Multimedia Applications

Relevance To Conference: Multi-view clustering endeavors to explore consistency and complementary information from a substantial volume of unlabeled multi-view/multi-modal data, constituting an exceptionally vital subfield within the realm of multimodal fusion. In this work, we target a common scenario in the real world: where multi-modal data is distributed across multiple clients. To accommodate this scenario, we propose HFMVC, a heterogeneity-aware federated deep multi-view clustering scheme. It achieves better clustering results through collaboration among multiple clients while safeguarding data privacy. We propose a dual contrastive learning mechanism to enhance the consistency of multi-view data, enabling rapid convergence and knowledge sharing among clients. Besides, We reveal the characteristics of feature representations in diverse heterogeneous environments and introduce a heterogeneity-aware module. This enables HFMVC to selectively adjust clustering strategies for robust adaptability, whether in IID or Non-IID scenarios. Extensive experiments validate the superior performance of HFMVC. For instance, on the MNIST-USPS dataset, HFMVC outperforms the state-of-the-art (SOTA) method by 36.83\% to 64.91\% in ACC, 41.39\% to 64.39\% in NMI, and 50.28\% to 79.06\% in ARI.

Submission Number: 3321