Anthropocentric bias and the possibility of artificial cognition

Raphaël Millière, Charles Rathkopf

Keywords: language models, psychology, anthropomorphism, anthropocentrism, comparative bias, mechanistic interpretability

TL;DR: To accurately assess the cognitive capacities of large language models, we must overcome anthropocentric biases by empirically investigating LLM-specific competencies and mechanisms.

Abstract: Evaluating the cognitive capacities of large language models (LLMs) requires overcoming not only anthropomorphic but also anthropocentric biases. This article identifies two types of anthropocentric bias that have been neglected: overlooking how auxiliary factors can impede LLM performance despite competence (Type-I), and dismissing LLM mechanistic strategies that differ from those of humans as not genuinely competent (Type-II). Mitigating these biases necessitates an empirically-driven, iterative approach to mapping cognitive tasks to LLM-specific capacities and mechanisms, which can be done by supplementing carefully designed behavioral experiments with mechanistic studies.

Submission Number: 66