Deep Active Learning with Manifold-Preserving Trajectory Sampling

chenhao wang, Rong Fu, Simon James Fong, DOU JIABAO, Qiaosong Gao, Nishanth Artham, Vijaya Sindhoori Kaza, Yingrui Ji

Keywords: Active learning, Manifold, MMD, Stochastic Weight Averaging

Abstract: Active learning (AL) is for optimizing the selection of unlabeled data for annotation (labeling), aiming to enhance model performance while minimizing labeling effort. The key question in AL is which unlabeled data should be selected for annotation. Existing deep AL methods arguably suffer from bias incurred by labeled data, which takes a much lower percentage than unlabeled data in AL context. We observe that such an issue is severe in different types of data, such as vision and non-vision data. To address this issue, we propose a novel method, namely Manifold-Preserving Trajectory Sampling (MPTS), aiming to enforce the feature space learned from labeled data to represent a more accurate manifold. By doing so, we expect to effectively correct the bias incurred by labeled data, which can cause a biased selection of unlabeled data. Despite its focus on manifold, the proposed method can be conveniently implemented by performing distribution mapping with MMD (Maximum Mean Discrepancies). Extensive experiments on various vision, non-vision and video benchmark datasets demonstrate the superiority of our method.

Primary Area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Submission Number: 9500