Uncertainty Based Active Learning Strategy for Interactive Weakly Supervised Learning through Data Programming

Anonymous

Keywords: active learning, weakly supervised learning, data programming, labeling cost

TL;DR: We propose and evaluate an uncertainty-based active learning strategy for interactive weakly supervised learning to reduce labeling cost without sacrificing interpretability of data.

Abstract:

Easy to build and reliable machine learning models are what all data analysts want. Although machine learning is advancing daily, the labeling cost for supervised learning and the black-box nature of machine learning are the main obstacles to its further diffusion. As a method of reducing the labeling cost without increasing the black-box nature, weakly supervised learning, especially data programming, is gaining attention. Its advantage is due to labeling functions that domain experts create based on their knowledge instead of labeling each data point manually. However, data programming alone cannot reduce the actual process cost.
This is because domain experts have to carry out a full search in their mind and endlessly implement labeling functions without any insight into what unimplemented labeling functions will be effective. We propose an active learning strategy for interactive weakly supervised learning with labeling functions to solve this problem. The proposed method iteratively presents a small number of highly prioritized data points to be labeled by additional labeling functions considering the uncertainty of predictions. With this method, domain experts need to only implement their knowledge that can be applied to a small number of presented data points as a labeling function. We also verified the effectiveness of this method through a six-class text classification task. The experimental results indicate the effectiveness of the method and its high potential in a machine learning implementation.