Batch Reinforcement Active Learning with TrustSet Extraction
Primary Area: general machine learning (i.e., none of the above)
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Keywords: Active Learning; Reinforcement Learning; TrustSet
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Abstract: Data labeling has nontrivial cost especially when large deep learning models are data hungry. Batch active learning is a common approach to increase label efficiency by selecting informative data batch for oracle to annotate, with the purpose that training target model on selected subset could yield the performance from training with full dataset. Uncertainty and diversity are the main concern for data sampling, but it is difficult to clearly analyze them before knowing data labels. Recent batch active learning leverages pretrained model to extract feature space of dataset and measure uncertainty and diversity of sampled data points by expert knowledge such as Mahalanobis Distance. Drawbacks of these methods refer to the lack of usage of feedback from selected data and uncleared relationship between feature distribution and actual accuracy achieved by target model. To deal with this issue, we propose TrustSet, which is extracted from labeled dataset containing most important data for training data sampler. Compared with analysis from unlabeled data pool, with ground truth knowledge, TrustSet is more reliable and correlated to target model accuracy. Then we formalize batch active learning as a reinforcement learning problem. Taking TrustSet as goal, we train a policy to adaptively sample data with high potential to be within TrustSet of unlabeled data pool. By evaluating on 10 image classification benchmarks, proposed active learning method achieves new state-of-the-art result.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 1456
Loading