The Cultural Trilemma: Disentangling the Trade-offs between Accuracy, Authenticity, and Neutrality in LLM Storytelling
Keywords: large language models, cultural bias, cultural alignment, narrative generation, representational harm, hallucination analysis, code-switching, multilingual evaluation, social bias auditing, synthetic evaluation datasets
Abstract: As Large Language Models (LLMs) are increasingly deployed globally, their ability to navigate diverse cultural narratives with fidelity is critical. This paper presents a comprehensive audit of 10 state-of-the-art LLMs (including Gemini 2.0, DeepSeek v3, and Llama 4), analyzing a novel dataset of 37,080 narratives generated across 206 cultures. We systematically evaluate performance across 18 dimensions, revealing a fundamental "Cultural Trilemma'': current models fail to simultaneously optimize for factual grounding, creative authenticity, and cultural neutrality. We find that highly aligned models (e.g., GPT-3.5) achieve low hallucination rates (<1%) but suffer from severe "cultural flattening,'' suppressing native code-switching and defaulting to Western norms for universal events. Conversely, models excelling in linguistic authenticity (e.g., Mistral Large) exhibit significantly higher rates of concept bleeding and inconsistency. Furthermore, we quantify a persistent Exoticism Bias, where Indigenous cultures are disproportionately described as "ancient'' (Score > 1.1) compared to Western cultures, and a strict Male Default in occupational roles (e.g., 91% of "Farmers'' are depicted as male). We conclude that current safety alignment imposes a tax on cultural depth, necessitating new ``thick'' evaluation paradigms for non-hegemonic identities.
Paper Type: Long
Research Area: Computational Social Science, Cultural Analytics, and NLP for Social Good
Research Area Keywords: misinformation detection and analysis, language/cultural bias analysis
Contribution Types: Model analysis & interpretability, NLP engineering experiment, Data resources, Data analysis
Languages Studied: English
Submission Number: 5761
Loading