Learning through Adaptive Queries: a Directional Derivative Approach

Doyoung Heo, Hwan Kyu Sung, Sanghwa Kim, Seungki Min

Keywords: Hybrid Offline-to-Online Learning, Bayesian Active Learning and Adaptive Query Selection, Active Learning and Bandits

Abstract:

In the task of Bayesian active learning, a learner aims to identify unknown model parameters by asking a series of queries in an adaptive manner.
We consider tasks with binary responses and suggest a simple adaptive query selection rule, called the Directional Uncertainty Reduction Maximization (DURM) algorithm.
The algorithm utilizes \emph{directional derivative} of the likelihood function as a proxy for the informativeness of an observation along a particular direction in the parameter space.
In each query selection step, it identifies the currently most uncertain direction based on Laplace's approximation and then selects the query with the largest directional derivative along that direction.
By doing so, it collects a new observation whose information gain is best aligned with the direction that the information is most needed.
We investigate the algorithm's behavior and prove its optimality theoretically for two canonical response models, the logistic model, and the MIRT model.
Our numerical experiments confirm these findings using synthetic data and further demonstrate our algorithm's effectiveness using real-world data.

Submission Number: 31