The MENAValues Shared Task on Cultural and Multilingual Alignment

Pardis Sadat Zahraei, Ehsaneddin Asgari

Keywords: Cultural Alignment, MENA Region, Multilingual NLP, Value Alignment

TL;DR: A competition to develop LLMs that accurately reflect Middle Eastern and North African cultural values across multiple languages and contexts.

Abstract: This proposal outlines a shared task on evaluating and improving the cultural alignment of Large Language Models (LLMs) with the values of the Middle East and North Africa (MENA) region. The competition is based on the MENAValues Benchmark, a novel dataset derived from large-scale, authoritative human surveys. Participants will be challenged to develop models that not only accurately reflect the documented values of MENA populations but also maintain consistency across different languages and contextual framings. The task aims to foster innovation in creating more culturally aware and globally aligned AI systems, addressing a critical gap in current evaluation efforts. This proposal details the problem statement, the ethically sourced dataset, robust evaluation criteria, a strong baseline model, and a comprehensive plan for execution and publication.

Submission Number: 82