Exploring Large Language Models' World Perception: A Multi-dimensional Evaluation through Data Distribution

ACL ARR 2025 February Submission7961 Authors

Abstract: In recent years, large language models (LLMs) have achieved remarkable success across diverse natural language processing tasks. Nevertheless, their understanding of core human experiences remains underexplored. Current benchmarks for LLM evaluation typically focus on a single aspect of linguistic understanding, thus failing to capture the full breadth of its abstract reasoning about the world. To address this gap, we propose a multidimensional paradigm to investigate the capacity of LLMs to perceive the world through temporal, spatial, emotional, and causal aspects. We conduct extensive experiments by partitioning datasets according to different distributions and employing various prompting strategies. Our findings reveal significant differences and shortcomings in how LLMs handle temporal granularity, multi-hop spatial reasoning, subtle emotions, and implicit causal relationships. While sophisticated prompting approaches can mitigate some of these limitations, substantial challenges persist in effectively capturing abstract human perception. We aspire that this work, which assesses LLMs from multiple perspectives of human understanding of the world, will guide more instructive research on the LLMs’ perception or cognition. The data and code will be released soon.

Paper Type: Long

Research Area: Interpretability and Analysis of Models for NLP

Research Area Keywords: Interpretability and Analysis of Models for NLP, Computational Social Science and Cultural Analytics, Generation

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 7961