Enhancing Human-in-the-Loop Learning for Binary Sentiment Word Classification

Belen Martin-Urcelay, Christopher J. Rozell, Matthieu R. Bloch

Abstract: While humans intuitively excel at classifying words according to their connotation, transcribing this innate skill into algorithms remains challenging. We present a human-guided methodology to learn binary word sentiment classifiers from fewer interactions with humans. We introduce a human perception model that relates the perceived sentiment of a word to the distance between the word and the unknown classifier. Our model informs the design of queries that capture more nuanced information than traditional queries solely requesting labels. Together with active learning strategies, our approach reduces human effort without sacrificing learning fidelity. We validate our method through experiments with human data, demonstrating improved accuracy in binary sentiment word classification.