How Do LLMs Ask Questions? A Pragmatic Comparison with Human Question-Asking

Chani Jung, Jimin Mun, Xuhui Zhou, Alice Oh, Maarten Sap, Hyunwoo Kim

Keywords: Question asking, pragmatics, large language models

Abstract: Question asking is a key linguistic and cognitive skill that supports collaboration and diverse social actions. However, large language models (LLMs) often underuse questions in their outputs, leading to misunderstandings or unproductive outputs. Toward bridging this gap, we analyze human questions from a real-world social setting---Reddit---and compare them to LLM-generated questions. We use a pragmatics-based taxonomy of social actions to examine six open- and closed-source model families. Our analysis shows that LLMs fall short of capturing the diversity and balance of human question-asking, with significant differences in question type distributions. Prompting often introduces prompt-specific biases that diverge from human patterns, while the effects of instruction-tuning are model-dependent and inconsistent across social functions. These findings highlight the need for more fine-grained approaches to align LLMs with human-like questioning behavior.

Submission Number: 35