A Multimodal Approach for Hate and Offensive Content Detection in Tamil: From Corpus Creation to Model Development

Jayanth Mohan, Mekapati Spandana Reddy, Premjith B, JYOTHISH LAL G, Bharathi Raja Chakravarthi

Abstract: Detecting hate speech on social media platforms is vital to mitigate technology-facilitated violence (TFV). Extensive research has targeted widely spoken languages like English, but there’s a notable gap in studying hate speech detection in low-resource languages like Tamil. Additionally, with social media platforms now supporting various modalities, including text, speech, and video, effective techniques for hate speech detection in multimodal formats, especially videos, are crucial. However, detecting hate speech in Tamil presents unique challenges due to its morphology and code-mixing nature. This paper presents a comprehensive approach for detecting hate speech in Tamil, with a focus on multimodal data. We introduce a new dataset, the Multimodal Tamil Hate (MATH) dataset, comprising videos along with their audio and textual transcripts, annotated with four categories of hate speech: offensive, sexist, racist, and casteist. To classify hate speech categories, we leverage transformer-based models. Through a series of experiments, we evaluated the performance of each modality individually and explored their fusion using multimodal approach. BERT-based models were used for textual analysis to extract informative features, the TimeSformer model was employed for video modality, and Wav2vec2 was used for audio modality. Specifically, we attained 81.82% accuracy and a 68.65% F1 score for the text modality, 63.63% accuracy and a 50.60% F1 score for the audio modality, and 45.45% accuracy and a 33.64% F1 score for the video modality. By integrating optimal combinations of models from each modality and employing machine learning classifiers, we achieved an accuracy of 81.82% and an F1 score of 66.67% in our hate speech classification task. Our research findings highlight the effectiveness of employing a multimodal approach for hate speech detection in Tamil, showcasing its efficacy in curbing the dissemination of hateful content on social media platforms.