Learning to Ask for Conversational Machine Learning

Shashank Srivastava, Igor Labutov, Tom Mitchell

Abstract:

Natural language has recently been increasingly explored as a medium of supervision for
training machine learning models. Here, we
explore learning classification tasks using language in a conversational setting – where the
automated learner does not simply receive language input from a teacher, but can proactively
engage the teacher by asking template-based
questions. We experiment with a reinforcement learning framework, where the learner’s
actions correspond to question types and the
reward for asking a question is based on how
the teacher’s response changes performance
of the resulting machine learning model on
the learning task. In this framework, learning
good question-asking strategies corresponds to
asking sequences of questions that maximize
the cumulative (discounted) reward, and hence
quickly lead to effective classifiers. Empirical
analysis shows that learned question-asking
strategies can expedite classifier training by
asking appropriate questions at different points
in the learning process. The approach allows
learning using a blend of strategies, including
learning from observations, explanations and
clarifications.