When More is not Necessary Better: Multilingual Auxiliary Tasks for Zero-Shot Cross-Lingual Transfer of Hate Speech Detection Models

Anonymous

Abstract: Zero-shot cross-lingual transfer learning has been shown to be highly challenging for tasks involving a lot of linguistic specificities or when a cultural gap is present between languages, such as in hate speech detection. In this paper, we highlight this limitation on several datasets and investigate how training on multilingual auxiliary tasks -- sentiment analysis, named entity recognition, and tasks relying on syntactic information -- impacts the zero-shot transfer of hate speech detection models across languages. We show the positive impact of these tasks, particularly named entity recognition, for bridging the gap between languages. Then, we present cases where the language model training data prevents hate speech detection models from benefiting from a knowledge proxy brought by auxiliary tasks fine-tuning. Our results warrant further investigation on how to best address cultural gap issues in resource-scarce scenario.

Paper Type: long