Is neural always better? SMT versus NMT for Dutch text normalization

Claudia Matos Veliz, Orphée De Clercq, Véronique Hoste

Abstract: Highlights • The Statistical Machine Translation approach achieved the best results. • Data augmentations increased the number of errors for the neural approach. • Byte-pair encoding is not viable when dealing with low-resources environments. • The CopyNet algorithm drastically reduces the number of overnormalizations. • The CopyNet algorithm is capable of correcting around 87% of the required edits. Abstract Social media texts have become one of the most used forms of written language and a valuable source of information for companies. However, despite the wealth of information hidden in social media language for profiling, information extraction and sentiment analysis purposes, this user-generated content is also characterized by non-standard language. This makes it difficult to process using standard NLP techniques as these are typically trained on standard text. Text normalization is often used as a preprocessing step to overcome this problem. In this work, we take a machine translation perspective on text normalization and investigate both statistical and neural methods. We perform text normalization in a low-resource scenario and report both SMT and NMT experiments for three different Dutch user-generated text types: tweets, message board posts and text messages. Furthermore, we look into overcoming the over-normalization problem in NMT using different techniques, viz. teacher forcing, copy mechanism and byte pair encoding. Our results reveal that even though the SMT approach obtains the best results, the NMT system using CopyNet shows promising results even in this low-resource setting, solving more normalization issues than SMT although not solving all the over-normalization problems.