Bounds on the Generalization Error in Active Learning

Vincent Menden, Yahya Saleh, Armin Iske

Keywords: Active Learning, Empirical Risk Minimization Principle, Integral Probability Metric

Abstract:

We establish empirical risk minimization principles for active learning by
deriving a family of upper bounds on the generalization error. Aligning with empirical observations, the bounds suggest that superior query algorithms can be obtained by
combining both informativeness and representativeness query strategies, where the latter is assessed using integral probability metrics.
To facilitate the use of these bounds in
application, we systematically link diverse active
learning scenarios, characterized by their loss functions and hypothesis
classes to their corresponding upper bounds. Our results show that
regularization techniques used to constraint the complexity of various hypothesis
classes are sufficient conditions to ensure the validity of the bounds.
The present work enables principled
construction and empirical quality-evaluation of query algorithms in active learning.

Submission Number: 18