Online Active Learning with Surrogate Loss Functions

Giulia DeSalvo, Claudio Gentile, Tobias Sommer Thune

Keywords: active learning, streaming, weak labels

TL;DR: We introduce a novel active learning algorithm with provable guarantees that works with surrogate loss functions and achieving compelling experimental performances

Abstract:

We derive a novel active learning algorithm in the streaming setting for binary classification tasks. The algorithm leverages
weak labels to minimize the number of label requests, and trains a model to optimize a surrogate loss on a resulting set of labeled and
weak-labeled points. Our algorithm jointly admits two crucial properties: theoretical guarantees in the general agnostic setting and a strong empirical performance. Our theoretical analysis shows that the algorithm attains favorable generalization and label complexity bounds, while our empirical study on 18 real-world datasets demonstrate that the algorithm outperforms standard baselines, including the Margin Algorithm, or Uncertainty Sampling, a high-performing active learning algorithm favored by practitioners.

Code Of Conduct: I certify that all co-authors of this work have read and commit to adhering to the NeurIPS Statement on Ethics, Fairness, Inclusivity, and Code of Conduct.