Building Bridges: A Dataset for Evaluating Gender-Fair Machine Translation into German

Anonymous

Abstract: The translation of gender-neutral person terms (e.g., the students) is often non-trivial. An interesting case poses the translation from English to German – in German, every noun is gendered, and if the gender of the referent(s) is unknown or diverse, the generic masculine (die Studenten (m.)) is commonly used. This, however, reduces the visibility of other genders, such as women and non-binary people. To counteract gender discrimination, a societal movement towards using gender-fair language exists (e.g., by adopting neosystems). However, gender-fair German is currently barely supported in Machine Translation (MT), requiring costly post-editing or manual translations. We address this research gap by studying gender-fair language in English to German MT. Concretely, we enrich a community-created gender-fair language dictionary, and sample multi-sentence test instances from encyclopedic text and parliamentary speeches. Using these novel resources, we conduct the first benchmark study involving two commercial systems and six neural MT models for translating words in isolation and words in larger contexts across two domains. Our findings show that most systems produce mainly masculine forms, and rarely gender-neutral variants, high- lighting the need for future research

Paper Type: short

Research Area: Ethics, Bias, and Fairness

Contribution Types: Data resources, Data analysis

Languages Studied: English, German