AFMCC: Asynchronous Federated Multi-modal Constrained Clustering

ICLR 2026 Conference Submission8460 Authors

Keywords: Federated clustering, multimodal learning, data heterogeneity, modality missingness, asynchronous

TL;DR: This paper introduces AMCFC, an Asynchronous Multi-modal Constrained Federated Clustering framework designed to address challenges in multimodal clustering, including data heterogeneity, modality missingness.

Abstract: Federated multi-modality clustering (FedMMC) aims to cluster distributed multi-modal data without compromising privacy. Existing approaches often rely on contrastive learning (CL), but suffer from representation degeneration, arbitrary modality missing, and computational imbalance. We propose Asynchronous Federated Multi-modal Constrained Clustering (AFMCC), which tackles these challenges through three key designs: (i) a Class-Correlation Matrix (CCM) regularization to prevent CL degeneration and enhance cluster separability, (ii) client-specific weighted aggregation to handle modality heterogeneity, and (iii) a weighted asynchronous aggregation strategy to mitigate computational imbalance and accelerate convergence. We further provide a theoretical analysis of AFMCC through a particle dynamics lens. Extensive experiments on diverse benchmarks demonstrate that AFMCC consistently outperforms state-of-the-art FedMMC methods in clustering accuracy and efficiency, while preserving privacy. We have released the source code and the dataset as supplementary material.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 8460