Veridicality Beyond Factuality: Turkish Evidentiality as a Test of Human and LLM Reasoning

Sercan Karakas

Keywords: evidentiality, veridicality, source trustworthiness, large language models

TL;DR: Humans use source trustworthiness to guide Turkish evidential choice, but current LLMs do not, revealing a gap in trust-sensitive veridicality and epistemic grounding.

Abstract: This study investigates whether humans and large language models are sensitive to source trustworthiness when choosing Turkish evidential morphology. Turkish past tense requires a choice between -DI, often associated with stronger/direct commitment, and -mIş, associated with indirect or weaker commitment. The paper asks whether this choice is shaped not only by information source type, but also by how trustworthy that source is. To test this, the authors designed a controlled cloze task in which the source is always explicitly external (“according to X”) but varies in credibility, for example an official website versus an informal message. In a human experiment with 75 native speakers of Turkish, high-trust contexts significantly increased -DI responses, while low-trust contexts increased -mIş responses. Low-trust contexts also led to slower response times, suggesting greater processing difficulty or reduced confidence. These findings support a trust-sensitive veridicality account, where source reliability modulates linguistic commitment. The paper then evaluates 11 LLMs on a 40-item human-matched dataset and a larger 200-item LLM-only dataset using log-probability scoring over paired -DI / -mIş continuations. Unlike humans, no individual model and no pooled analysis showed reliable trust sensitivity. Instead, model preferences were largely driven by tokenization artifacts and checkpoint-specific biases. The results reveal a clear human–LLM gap in epistemic grounding and source-sensitive evidential reasoning.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 93