ICAN: Introspective Convolutional Attention Network for Semantic Text Classification

Sounak Mondal, Suraj Modi, Sakshi Garg, Dhruva Das, Siddhartha Mukherjee

Abstract: Semantic text classification involves a deep understanding of natural language by going beyond mere syntactic information and capturing complex semantic properties like synonymy, polysemy and negation. We propose a novel attention mechanism called Introspective Semantic Attention embedded within a cascaded CNN architecture. We call our network Introspective Convolutional Attention Network (ICAN). In addition to extracting semantic information using convolution operations, ICAN derives semantic attention from its primary convolutional features instead of using a separate attention module. We also introduce a novel hybrid pooling strategy for our architecture which aids in preserving pertinent information encapsulated within a sentence, while discarding meaningless noise. Our architecture, while light-weight and efficient, promises high accuracy with respect to state of the art architectures - making it ideal for embedded systems and commercial servers alike.