Desiderata for Representation Learning from Identifiability, Disentanglement, and Group-Structuredness

Hamza Keurti, Patrik Reizinger, Bernhard Schölkopf, Wieland Brendel

Keywords: Representation Learning, Disentanglement, Group Theory, Identifiability, Group-Structured Representations

TL;DR: We highlight the shortcomings of disentanglement for both statistical representations and group-structured representations and show they have complementary contributions to the critera of a good representation.

Abstract:

Machine learning subfields define useful representations differently: disentanglement strives for semantic meaning and symmetries, identifiability for recovering the ground-truth factors of the (unobservable) data generating process, group-structured representations for equivariance.
We demonstrate that despite their merits, each approach has shortcomings. Surprisingly, joining forces helps overcome the limitations: we use insights from latent space statistics, geometry, and topology in our examples to elucidate how combining the desiderata of identifiability, disentanglement, and group structure yields more useful representations.

Type Of Submission: Extended Abstract (4 pages, non-archival)

Submission Number: 88