Fast and Accurate Text Classification: Skimming, Rereading and Early StoppingDownload PDF

15 Feb 2018 (modified: 21 Mar 2023)ICLR 2018 Conference Blind SubmissionReaders: Everyone
Abstract: Recent advances in recurrent neural nets (RNNs) have shown much promise in many applications in natural language processing. For most of these tasks, such as sentiment analysis of customer reviews, a recurrent neural net model parses the entire review before forming a decision. We argue that reading the entire input is not always necessary in practice, since a lot of reviews are often easy to classify, i.e., a decision can be formed after reading some crucial sentences or words in the provided text. In this paper, we present an approach of fast reading for text classification. Inspired by several well-known human reading techniques, our approach implements an intelligent recurrent agent which evaluates the importance of the current snippet in order to decide whether to make a prediction, or to skip some texts, or to re-read part of the sentence. Our agent uses an RNN module to encode information from the past and the current tokens, and applies a policy module to form decisions. With an end-to-end training algorithm based on policy gradient, we train and test our agent on several text classification datasets and achieve both higher efficiency and better accuracy compared to previous approaches.
TL;DR: We develop an end-to-end trainable approach for skimming, rereading and early stopping applicable to classification tasks.
Keywords: Topic Classification, Sentiment Analysis, Natural Language Processing
Data: [AG News](, [IMDb Movie Reviews](
13 Replies