Online Active Learning with Surrogate Loss FunctionsDownload PDF

21 May 2021, 20:48 (edited 11 Jan 2022)NeurIPS 2021 SpotlightReaders: Everyone
  • Keywords: active learning, streaming, weak labels
  • TL;DR: We introduce a novel active learning algorithm with provable guarantees that works with surrogate loss functions and achieving compelling experimental performances
  • Abstract: We derive a novel active learning algorithm in the streaming setting for binary classification tasks. The algorithm leverages weak labels to minimize the number of label requests, and trains a model to optimize a surrogate loss on a resulting set of labeled and weak-labeled points. Our algorithm jointly admits two crucial properties: theoretical guarantees in the general agnostic setting and a strong empirical performance. Our theoretical analysis shows that the algorithm attains favorable generalization and label complexity bounds, while our empirical study on 18 real-world datasets demonstrate that the algorithm outperforms standard baselines, including the Margin Algorithm, or Uncertainty Sampling, a high-performing active learning algorithm favored by practitioners.
  • Supplementary Material: pdf
  • Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.
13 Replies