Do LLMs Know Spring is Green? A Synesthesia Study of LLMs Response

ACL ARR 2025 May Submission7335 Authors

Abstract: Large Language Models (LLMs) exhibit emergent capabilities beyond core language tasks, and demonstrate certain cognitive alignment between humans. However, the potential of LLMs to simulate human-like cross-modal cognitive alignments, such as synesthesia, remains unexplored. Synesthesia involves consistent associations between concepts and sensory experiences (e.g., linking numbers to colors), a phenomenon also reflected in cross-modal correspondences observed even in non-synesthetes. In this work, we conduct the study of whether modern LLMs replicate such synesthetic alignment by evaluating their responses on color association tasks across diverse conceptual domains: digits, letters, temporal concepts (e.g., days, months), spatial directions, and abstract entities. Using standardized prompts, we analyze responses from multiple LLMs and compare them to human data collected from 260 participants. Colors are mapped to a perceptually uniform space (CIELAB), with alignment quantified via the CIEDE2000 metric. Our results reveal that LLMs show significant alignment with human consensual patterns, particularly for temporal concepts like seasons and months, achieving color differences comparable to human variability. However, abstract concepts (e.g., directions) exhibit greater divergence. Cultural influences (e.g., Western vs. Chinese contexts) impact alignment, while gender differences in humans do not translate to LLMs. Model size and architecture also affect performance, with larger models demonstrating stronger alignment. These findings highlight LLMs’ ability to capture certain cross-modal associations, offering insights into their implicit grounding of abstract concepts and implications for multimodal applications requiring sensory-conceptual integration.

Paper Type: Long

Research Area: Special Theme (conference specific)

Research Area Keywords: cognitive modeling; computational psycholinguistics; human-subject application-grounded evaluations

Contribution Types: Data analysis

Languages Studied: English; Chinese

Keywords: cognitive modeling, computational psycholinguistics, human-subject application-grounded evaluations

Submission Number: 7335