CuPS: Measuring Cultural Preference Signatures in LLM/VLM Agents and Their Steering by Profile Memories

Kyeong Seon Kim, GeonU Kim, Joohyun Chang, Hyeyeon Kim, Tae-Hyun Oh

Keywords: LLM/VLM agents show measurable cultural preference profiles on ambiguous tasks, and persistent profile memories can shift those preferences even when the profile contains only indirect user cues.

TL;DR: cultural AI, language models, vision-language models, agent memory, cultural preference profiling, multimodal evaluation, personalization

Abstract: Cultural background shapes how people read the same signal differently. In this context, we ask a simple question. Do LLM/VLM agents also read these signals differently? We call this a cultural preference signature. We further ask whether this signature can be shifted by user information contained in pre-execution instruction documents that agents commonly consult, such as memory.md or agent.md. We introduce CuPS, a benchmark designed to measure such signatures. CuPS covers gesture interpretation, triadic categorization, and time-space mapping, with each domain measured across input forms that agents can receive, including text, emoji tokens, and rendered emoji images. Across Qwen and Llama agents, we observe that, much like people, each model carries its own way of reading these signals. In profile-memory experiments, the initial signature shifts in country-specific ways depending on user information documents constructed from personas sampled from NVIDIA Nemotron-Personas. These country-specific shifts appear not only when the user information is given explicitly, but also when it is given only implicitly.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 73