Weak Supervision Variational Auto-Encoder

Francesco Tonolini, Nikolaos Aletras, Yunlong Jiao, Gabriella Kazai

Keywords: Variational Auto-Encoders, Weak Supervision, Weak Labelling

TL;DR: A VAE model with specifically designed components to perform weak supervision. Compared to existing weak supervision methods, it is considerably more robust to labelling functions design.

Abstract:

Recent advances in weak supervision (WS) techniques allow to mitigate the enormous labelling cost of human data annotation for deep learning by automating it using simple rule-based labelling functions (LFs). However, LFs need to be carefully designed, often requiring expert domain knowledge to be of sufficient accuracy, cover enough data and be independent of each other for existing WS methods to be viable. In addition, weak supervision methods often rely on small amounts of validation data with true labels to fine-tune and select models.
To tackle these issues, we propose the Weak Supervision Variational Auto-Encoder (WS-VAE), a novel framework that combines unsupervised representation learning and weak labelling to reduce the dependence of WS on expert and manual engineering of LFs. The proposed technique learns from inputs and weak labels jointly and captures the input signals distribution with an artificial latent space, leading to considerably improved robustness to LFs quality. Our extensive empirical evaluation shows that our WS-VAE performs competitively to existing WS on a standard WS benchmark while it is substantially more robust to LF engineering.

Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.

No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.

Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics

Submission Guidelines: Yes

Please Choose The Closest Area That Your Submission Falls Into: Deep Learning and representational learning