Adaptive Data Collection for Latin-American Community-sourced Evaluation of Stereotypes (LACES)

ACL ARR 2026 January Submission3972 Authors

Keywords: Ethics, Bias, and Fairness, Resources and Evaluation, Human-Centered NLP, Multilingualism and Cross-Lingual NLP

Abstract:

The evaluation of societal biases in NLP models is critically hindered by a geo-cultural gap, This leaves regions such as Latin America severely underserved, making it impossible to adequately assess or mitigate the perpetuation of harmful regional stereotypes in language technologies.

This paper presents LACES, a stereotype association dataset, for 15 Latin American countries. This dataset includes 4,789 stereotype associations manually created and annotated by 83 participants. The dataset was developed through targeted community partnerships across Latin America.

Additionally, in this paper, we propose a novel adaptive data collection methodology that uniquely integrates the sourcing of new stereotype entries and the validation of existing data within a single, unified workflow. This approach results in a resource with more unique stereotypes than previous static collection methods, enabling a more efficient stereotype collection. The paper further supports the quality of LACES by demonstrating reduced efficacy of debiasing methods on this dataset in comparison to existing popular stereotype benchmarks.

Paper Type: Long

Research Area: Ethics, Bias, and Fairness

Research Area Keywords: Ethics, Bias, and Fairness, Resources and Evaluation, Human-Centered NLP, Multilingualism and Cross-Lingual NLP

Contribution Types: Model analysis & interpretability, Data resources, Data analysis

Languages Studied: Spanish, English

Submission Number: 3972