PRISM: A Principled Framework for Supervised Disentanglement via Bipartite Factorization

Kacper Kubicki, Krzysztof Ślot

Keywords: supervised disentanglement, representation learning, generative modeling, autoencoders, deep learning

Abstract: Learning structured representations that partition information based on its semantic contents remains a central challenge in deep generative modeling. In light of the established theoretical impossibility of purely unsupervised disentanglement, we address the pragmatic and well-posed objective of bipartite factorization: separating the single factor of variation corresponding to the supervisory label from all other residual sources of variation. We introduce a principled framework that achieves this separation through a learning mechanism that routes most of the intra-class variation into a class-agnostic latent subspace. The design of this mechanism is guided by a formal, information-theoretic analysis, which provides quantitative bounds on the learning outcome. We conduct a series of targeted experiments designed to validate the proposed mechanism, demonstrating its ability to produce a factorized representation with quantifiably low leakage of supervised information into the residual subspace, and illustrating the effectiveness of the resulting factorization on downstream tasks requiring precise latent control, such as targeted attribute swapping and manipulation of stylistic features.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 24342