Weakly-Supervised Disentangled Representation Learning via Filter-Based Adaptive Swapping

TMLR Paper6032 Authors

Abstract: Disentangled representation learning (DRL) aims to uncover semantically meaningful latent factors from observed data, thereby improving both interpretability and generalization of machine learning (ML) models. Despite remarkable progress, unsupervised DRL cannot achieve complete disentanglement without inductive biases or supervision. To address this challenge, existing approaches either rely on full supervision, which demands extensive manual labeling, or weak supervision, which involves complex training strategies that often result in unstable training. To address these limitations, we propose Filter-VAE, a weakly supervised variational autoencoder (VAE) that introduces a filter-based adaptive swapping strategy to learn stable and meaningful disentangled representations. Specifically, a relevance filter removes semantically meaningless latent factors, while an adaptive swapping filter exchanges those latent factors that have reached stability. With these two filters, Filter-VAE adaptively swaps only stable and semantically aligned latent factors, leading to robust and meaningful representations. We evaluate Filter-VAE on three standard benchmarks and our created traffic sign dataset in two downstream tasks: disentanglement and adversarial robustness. Experimental results demonstrate that Filter-VAE achieves strong disentanglement performance with reduced supervision and delivers remarkable robustness against diverse adversarial attacks and corruptions. The code will be released upon acceptance.

Submission Length: Regular submission (no more than 12 pages of main content)

Changes Since Last Submission:

1. Per reviewers' suggestion, we move the detailed algorithm to **Section 4.5**.
2. Per reviewers' suggestion, we update the math notation in **Section 3, 4, and 5**.
3. Per reviewers' suggestion, we remove the underscore in **Section 5.1.2**.
4. Per reviewers' suggestion, we add parentheses to all in-text citations.
5. Per reviewers' suggestion, we add real-world dataset experiments in **Appendix F**.
6. Per reviewers' suggestion, we have uploaded codes to **Supplementary Material**.

Assigned Action Editor: ~Yu_Yao3

Submission Number: 6032