Consonant is all you need: a compact representation of English text for efficient NLP

Maged S. Al-shaibani, Irfan Ahmad

Submission Type: Regular Long Paper

Submission Track: Efficient Methods for NLP

Submission Track 2: Language Modeling and Analysis of Language Models

Keywords: Text Representation, Tokenization, Language Modeling, Text Classification, POS Tagging, NER, NMT

TL;DR: A novel approach to represent English text in NLP using only consonants, which achieves comparable performance to standard representation while significantly reducing computational resources.

Abstract:

In natural language processing (NLP), the representation of text plays a crucial role in various tasks such as language modeling, sentiment analysis, and machine translation. The standard approach is to represent text in the same way as we, as humans, read and write. In this paper, we propose a novel approach to represent text with only consonants which presents a compact representation of English text that offers improved efficiency without sacrificing performance. We exploit the fact that consonants are more discriminative than vowels and by representing text using consonants, we can significantly reduce the overall memory and compute footprint required for storing and processing textual data.

We present two alternative representations: 'consonants-only', where we completely remove the vowels from the text, and 'masked-vowels', where we mask all the vowels into one special symbol. To evaluate our approaches, we conducted experiments on various NLP tasks, including text classification, part-of-speech (POS) tagging, named-entity recognition (NER), and neural machine translation (NMT), in addition to language modeling. Our results demonstrate that the proposed consonant-based representation achieves comparable performance compared to the standard text representation while requiring significantly fewer computational resources. Furthermore, we show that our representation can be seamlessly integrated with existing NLP models and frameworks, providing a practical solution for efficient text processing.

Last but not the least, we present a technique to retrieve the vowel information from our processed text representation keeping in mind the need to reproduce text in human readable form in some NLP applications.

Submission Number: 4369