Global Gallery: The Fine Art of Painting Culture Portraits through Multilingual Instruction Tuning

Anonymous

TL;DR: Studying the effects of multilingual instruction tuning on cultural understanding of LLMs.

Abstract:

Exploring the intersection of language and culture in Large Language Models (LLMs), this study critically examines their capability to encapsulate cultural nuances across diverse linguistic landscapes. Central to our investigation are three research questions: the efficacy of language-specific instruction tuning, the impact of pretraining on native language data, and the identification of optimal approaches to elicit accurate cultural knowledge from LLMs. Utilizing the GeoMLaMA benchmark for multilingual commonsense knowledge and an adapted CAMeL dataset (english-only) for evaluation of nuanced cultural aspects, our experiments span six different languages and cultural contexts, revealing the extent of LLMs' cultural awareness.
Our findings highlight a nuanced landscape: while language-specific tuning and bilingual pretraining enhance cultural understanding in certain contexts, they also uncover inconsistencies and biases, particularly in non-Western cultures. This work not only expands our understanding of LLMs' cultural competence but also emphasizes the importance of integrating diverse cultural perspectives in their development, aiming for a more globally representative and equitable approach in language modeling.

Paper Type: long

Research Area: Multilinguality and Language Diversity

Contribution Types: Model analysis & interpretability, Data resources

Languages Studied: English, Chinese, Hindi, Persian, Swahili, Greek