TactfulToM: Do LLMs have the Theory of Mind ability to understand White Lies?

ACL ARR 2025 May Submission4117 Authors

Abstract:

While recent studies explore Large Language Models' (LLMs) performance on Theory of Mind (ToM) reasoning tasks, research on ToM abilities require more nuanced social context is limited, such as white lies. We introduce TactfulToM, a novel English benchmark designed to evaluate LLMs' ability to understand white lies within real-life conversations and reason about prosocial motivations behind them, particularly used to spare others' feelings and maintain social harmony.
Our benchmark is generated through a multi-stage human-in-the-loop pipeline where LLMs expand manually designed seed stories into conversations to maintain the information asymmetry between participants necessary for authentic white lies. We show that Tactful ToM is challenging for state-of-the-art models, which perform below humans, revealing shortcomings in their ability to fully comprehend the ToM reasoning enabling understanding of white lies.

Paper Type: Long

Research Area: Resources and Evaluation

Research Area Keywords: LLMs, white lie, Thoery of Mind, social reasoning, benchmark, evaluation

Contribution Types: Model analysis & interpretability

Languages Studied: English

Submission Number: 4117