Transformer based multilingual joint learning framework for code-mixed and english sentiment analysis

Mamta, Asif Ekbal

Abstract: In recent times, there has been tremendous growth in the number of multi-lingual users on social media platforms. Consequently, the code-mixing phenomenon, i.e., mixing of more than one language, has become ubiquitous in Internet content. In this paper, we present a shared-private, multi-lingual, multi-task model coupled with a transformer-based pre-trained encoder for sentiment analysis of code-mixed and English languages. Our model is tailored for multitasking that transfers the knowledge between code-mixed and English sentiment tasks. We consider code-mixed sentiment analysis as the primary task and enhance its performance by English sentiment analysis (auxiliary task) by sharing knowledge between them. We fine-tune the Bidirectional Encoder Representation using Transformer (BERT) encoder in a shared-private fashion to obtain the shared and task-specific features using the multi-task objective function. We evaluate our proposed framework using three benchmark datasets for the Hindi-English (Hinglish), Punjabi-English (Punglish) code-mixed and English sentiment tasks. Experiment results justify that our proposed multi-task framework improves the performance of our primary task in comparison to the state-of-art single-task systems.