Regression with re-labeling for noisy data

Youngdoo Son, Seokho Kang

Abstract: Highlights • Novel active learning framework based on expectation-refinement sampling is presented. • It performs active learning with re-labeling for regression task with noisy annotator. • Exploration step selects unlabeled instance to be labeled next. • Refinement step labels again for labeled instance to improve label accuracy. • Experimental results demonstrate its effectiveness on benchmark datasets. Abstract Active learning, which focuses on building an accurate prediction model with a reduced cost by actively querying which instances should be labeled for training, has been successfully employed in several real-world applications involving expensive labeling costs. Although most existing active learning strategies have focused on labeling unlabeled instances, it has been shown that improving the quality of previously annotated labels is also important when the annotator produces noisy labels. In this study, we propose a novel active learning framework for regression, which is effective for the scenarios with noisy annotators, by providing a new sampling strategy named exploration-refinement (ER) sampling. The ER sampling performs two main steps: exploration and refinement. The exploration step involves finding unlabeled instances to be labeled, and the refinement step seeks to improve the accuracy of already labeled instances. The experimental results on several benchmark datasets demonstrate the effectiveness of the ER sampling with statistical significance.