Learning to Ask the Right Questions: A Multi-armed Bandits Approach

VIKAS DEEP, Achal Bassamboo, Sandeep Kumar Juneja, assaf zeevi

Abstract:

Adaptive testing where the hardness of the next question depends on the response of the candidate on
prior questions is used in a variety of settings. In this paper, we study the problem of designing an optimal
adaptive test with the goal to optimally classify a candidate’s ability into one of several categories or
grades. The candidate’s ability is considered an unknown factor, which, combined with the hardness of
the question, determines the chance of answering correctly. The learning algorithm is only able to observe
whether the candidate answers a given question correctly or not. We consider this problem from a fixed
confidence-based δ-correct framework. In our setting, this seeks to arrive at the correct grade of a given
candidate at the fastest possible rate, i.e., the fewest number of questions asked while guaranteeing that the
probability of error is less than a pre-specified and small δ. We derive a lower bound on the expected number
of questions asked for any sequential questioning strategy, which is a solution to a min-max optimization
problem. We develop geometrical insights into this optimization problem structure and its dual formulation.
In addition, we propose an algorithm that essentially matches these lower bounds. Our key conclusions are
that, asymptotically, any candidate needs to be asked questions at most at two (candidate ability-specific)
hardness levels, although, in reasonably general settings on the problem structure the questions that need
to be asked are at almost one hardness level. We also propose a related algorithm based on Gaussian
approximation that performs well numerically and admits suitable δ-correct performance guarantees in an
asymptotic regime.