ALBench: A Framework for Evaluating Active Learning in Object Detection

Zhanpeng Feng, Shiliang Zhang, Rinyoichi Takezoe, Wenze Hu, Manmohan Chandraker, Li-jia Li, Vijay K. Narayanan, Xiaoyu Wang

Abstract: Active learning is an important technology for automated machine learning systems. In contrast to Neural Architecture Search (NAS) which aims at automating neural network architecture design, active learning aims at automating training data selection process. It is especially critical for training a long-tailed task, in which positive samples are sparsely distributed. Active learning alleviates the expensive data annotation issue through incrementally training models powered with efficient data selection. Instead of annotating all unlabeled samples, it iteratively selects and annotates the most valuable samples. Active learning has been popular in image classification, but has not been fully explored in object detection. Most of current works on object detection are evaluated with different settings, making it difficult to fairly compare their performance. To facilitate the research in this field, this paper contributes an active learning benchmark framework named as ALBench for evaluating active learning in object detection. Developed on an automatic deep model training system, this ALBench framework is easy-to-use, compatible with different active learning algorithms, and ensures the same training and testing protocols. We hope this automated benchmark system helps researchers to easily reproduce literature's performance and have fair comparisons with prior arts.

Keywords: Active Learning, Deep Learning, Object Detection, Benchmark

One-sentence Summary: The paper proposed a benchmark system for active learning in object detection.

Track: Special track for systems, benchmarks and challenges

Reproducibility Checklist: Yes

Broader Impact Statement: Yes

Paper Availability And License: Yes

Code Of Conduct: Yes

Reviewers: Xiaoyu Wang, [e-mail]

CPU Hours: 0

GPU Hours: 0

TPU Hours: 0

Community Implementations: [ 2 code implementations](https://www.catalyzex.com/paper/albench-a-framework-for-evaluating-active/code)