Repairing Systematic Outliers by Learning Clean Subspaces in VAEs

Simao Eduardo, Kai Xu, Alfredo Nazabal, Charles Sutton

Keywords: variational autoencoder, deep generative models, outlier detection, data repair

Abstract:

Data cleaning often comprises outlier detection and data repair.
Systematic errors result from nearly
deterministic transformations that occur repeatedly in the data,
e.g. specific image pixels being set to default values or watermarks.
Consequently, models with enough capacity easily overfit to these
errors, making detection and repair difficult.
Seeing as a systematic outlier is a combination of patterns of a clean instance
and systematic error patterns, our main insight is that inliers can be
modelled by a smaller representation (subspace) in a model than outliers.
By exploiting this,
we propose \emph{Clean Subspace Variational Autoencoder (CLSVAE)},
a novel semi-supervised model for detection and automated repair of
systematic errors.
The main idea is to partition the latent space and model inlier and
outlier patterns separately.
CLSVAE is effective with much less labelled data compared to previous related
models, often with less than 2\% of the data.
We provide experiments using three image datasets in scenarios with
different levels of corruption and labelled set sizes, comparing to relevant baselines.
CLSVAE provides superior repairs without human intervention,
e.g. with just 0.25\% of labelled data we see a relative error
decrease of 58\% compared to the closest baseline.

One-sentence Summary: To repair outliers resulting from systematic errors, we propose a semi-supervised VAE that learns separate representations for inliers and systematic errors.