Fast and Accurate Text Classification: Skimming, Rereading and Early Stopping

Keyi Yu, Yang Liu, Alexander G. Schwing, Jian Peng

Keywords: Topic Classification, Sentiment Analysis, Natural Language Processing

TL;DR: We develop an end-to-end trainable approach for skimming, rereading and early stopping applicable to classification tasks.

Abstract: Recent advances in recurrent neural nets (RNNs) have shown much promise in many applications in natural language processing. For most of these tasks, such as sentiment analysis of customer reviews, a recurrent neural net model parses the entire review before forming a decision. We argue that reading the entire input is not always necessary in practice, since a lot of reviews are often easy to classify, i.e., a decision can be formed after reading some crucial sentences or words in the provided text. In this paper, we present an approach of fast reading for text classification. Inspired by several well-known human reading techniques, our approach implements an intelligent recurrent agent which evaluates the importance of the current snippet in order to decide whether to make a prediction, or to skip some texts, or to re-read part of the sentence. Our agent uses an RNN module to encode information from the past and the current tokens, and applies a policy module to form decisions. With an end-to-end training algorithm based on policy gradient, we train and test our agent on several text classification datasets and achieve both higher efficiency and better accuracy compared to previous approaches.

Data: [AG News](https://paperswithcode.com/dataset/ag-news), [IMDb Movie Reviews](https://paperswithcode.com/dataset/imdb-movie-reviews)