TAAL: Target-Aware Active Learning

Kunal Kotian, Indranil Bhattacharya, Shikhar Gupta, Kaushik Pavani, Naval Bhandari, Sunny Dasgupta

Abstract:

Pool-based active learning techniques have had success producing multi-class classifiers that achieve high accuracy
with fewer labels compared to random labeling. However, in an industrial setting where we often have class-level
business targets to achieve (e.g., 95% recall at 95% precision for each class), active learning techniques continue to
acquire labels for classes that have already met their targets, thus consuming unnecessary manual annotations.
We address this problem by proposing a framework called Target-Aware Active Learning that converts any active
learning query strategy into its target-aware variant by leveraging the gap between each class’ current estimated
accuracy and its corresponding business target. We show empirically that target-aware variants of state-of-the-art
active learning techniques achieve business targets faster on 2 open-source image classification datasets and 2
proprietary product classification datasets.