Extending the Contact Hypothesis: Cross-Linguistic Evaluation of Religion and Nationality Bias When Prompting LLMs in German and Icelandic

Ikae Catherine, Guðbjörg Linda Rafnsdóttir, Ragna Kemp Haraldsdóttir, Mascha Kurpicz-Briki

Track: Scientific Track

Keywords: Bias, Natural language processing, Language models

TL;DR: The paper demonstrates that bias in LLMs can be mitigated through positive contact prompts, and contrasts this with negative contact scenarios, which amplify biased responses.

Abstract:

Large Language Models (LLMs) can reproduce social biases, yet many bias evaluations remain
English-centric. We extend the Contact Hypothesis framework presented in previous work to German and Icelandic, focusing on religion and nationality. Evaluating GPT models (3.5, 4, 4-turbo, 4o, 5), we find that positive contact reduces biases in the answers of the LLMs, while negative contact amplifies it, with cross-
linguistic differences in magnitude and salience. Our results support the cross-linguistic robustness of contact-based probing and underscore the need for culturally contextualized evaluations. In addition to these insights, our contributions lies in the dataset that is made available on Github1 for further research.

Submission Number: 20