The Sky Is The Limit When Clustering Is Equated With Disentanglement

Thomas Kreutz, Max Mühlhäuser, Alejandro Sanchez Guinea

Keywords: Disentangled Representation Learning, Clustering, Factors of Variation, Variational Autoencoders (VAEs), Generative Models, Synthetic Image Data

Abstract: Disentangled representation learning allows data to be mapped to a latent space where factors of variation can be individually manipulated. These factors define a direct notion of similarity between observations that naturally groups them into clusters with shared factors of variation. While this has been empirically shown to be effective on simple datasets, it is unclear how or when complex real-world data can be disentangled into representations that allow the same degree of manipulation and clustering. To advance the field of disentangled representation learning and clustering, we provide a new theoretical perspective by equating disentanglement with clustering by using factors of variation as a measure of element-wise similarity. This leads to a simple yet important observation: Instead of explicitly clustering the elements of a dataset, we can implicitly cluster them by learning to represent and generate the elements of each cluster. Furthermore, this observation reveals that implicit clusters have a lower bound because (I) explicit clusters are a subset of implicit clusters, and (II) implicit clusters can generate novel elements not present in the finite dataset through combinatorial generalization. Building on these insights, we derive an implicit neural clustering approach based on identifying factors of variation in the latent space. We validate our findings through experiments on synthetic image data and empirical evidence from related state-of-the-art works. This demonstrates the practical relevance of our approach and promising potential for synthesizing complete datasets from limited data, addressing data distribution gaps, improving interpretability in cluster analysis, enhancing SSL and classification tasks, and reducing data storage space.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.

Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.

Reciprocal Reviewing: I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Submission Number: 11619