Red Teaming for Trust: Evaluating Multicultural and Multilingual AI Systems in Asia-Pacific

Akash Kundu, Adrianna Tan, Theodora Skeadas, Rumman Chowdhury, Sarah Amos

Track: Long Paper Track (up to 9 pages)

Keywords: AI Red Teaming, Multilingual Bias Detection, Fairness in LLMs

TL;DR: This paper introduces the first multicultural and multilingual AI Safety Red Teaming Challenge in the Asia-Pacific, revealing overlooked biases and advocating for more inclusive AI evaluations.

Abstract: This paper presents the first multicultural and multilingual AI Safety Red Teaming Challenge focused on the Asia-Pacific region, conducted in November and December 2024. Red teaming, a critical method for evaluating the safety and robustness of AI systems, involves stress-testing models to uncover vulnerabilities, biases, and limitations. While traditionally performed by AI developers in Western-centric contexts, this study expands the scope by emphasizing cultural and linguistic nuances unique to East, Southeast, and South Asia. The challenge included 54 participants from nine countries, representing academic and research institutions, and involved an in-person event followed by a virtual component. The primary objective was to establish a baseline for AI performance across diverse cultural and linguistic contexts, addressing the demographic and cultural disparities often overlooked in existing AI evaluations. Our findings underscore the necessity of addressing both universal and region-specific risks in AI, paving the way for more equitable global AI adoption.

Submission Number: 62