From What to Respond to When to Respond: Timely Response Generation for Open-domain Dialog Agents

ACL ARR 2026 January Submission9836 Authors

Keywords: open-domain dialogue system, evaluation and metrics, conversational modeling, automatic creation and evaluation of language resources, NLP datasets

Abstract:

While research on dialog response generation has primarily focused on generating coherent responses conditioning on textual context, the critical question of when to respond grounded on the temporal context remains underexplored.
To bridge this gap, we propose a novel task named timely dialog response generation and introduce the TimelyChat benchmark to evaluate two key aspects: response timing and time-conditioned responses, which focus on the capabilities of language models to predict appropriate delays and delayed responses.
Additionally, we construct a large-scale training dataset by leveraging unlabeled event knowledge from a temporal commonsense knowledge graph and employing a large language model (LLM) to synthesize 55K event-driven dialogs.
We then train Timer, a dialog agent designed to proactively predict time intervals and generate timely responses that align with those intervals.
Experimental results show that Timer outperforms instruction-tuned LLMs and other time-aware baselines in both turn-level and dialog-level evaluations.
We publicly release our data, model, and code.

Paper Type: Long

Research Area: Dialogue and Interactive Systems

Research Area Keywords: evaluation and metrics, conversational modeling

Contribution Types: Publicly available software and/or pre-trained models, Data resources

Languages Studied: English

Submission Number: 9836