Granular Information Bottleneck for Deep Multi-modal Clustering

Zhengzheng Lou, Mingyang Lv, Yuhan Zhan, Jin Qin, Chaoyang Zhang, Shizhe Hu

Keywords: Deep Multi-modal Clustering, Information Bottleneck

TL;DR: We propose a novel deep granular information bottleneck (DGIB) method for multi-modal clustering.

Abstract: Deep multi-modal clustering generally focuses on improving clustering accuracy by leveraging information from different modalities. However, existing methods are designed around the finest-grained points as input, which are neither efficient nor robust to noisy data, negatively affecting the clustering results. To this end, we propose a novel granular information bottleneck (GIB) for deep multi-modal clustering, which embeds a dual-tiered information bottleneck constraint mechanism operating synergistically at the granular and sample levels, thereby learning discriminative feature representations with enhanced inter-cluster separability. Specifically, GIB adaptively represents and covers the sample points through granular balls of different granularity levels, which effectively captures the feature distribution within each cluster. Simultaneously, information compression and preservation are used to exploit the independence and complementarity of modalities while optimizing cluster assignments alignment. Finally, the objectives of GIB are formulated as a target function based on mutual information, and we propose a variational optimization method to ensure its convergence. Extensive experimental results validate the effectiveness of the proposed GIB model in accuracy, reliability and robustness.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 3581