The PROPER Approach to Proactivity: Benchmarking and Advancing Knowledge Gap Navigation

ACL ARR 2026 January Submission7710 Authors

Keywords: Proactive Language Agents, Personalized AI Systems, Large Language Models, Knowledge Gap Detection

Abstract: Most language-based assistants follow a reactive \textit{ask-and-respond} paradigm, requiring users to explicitly state their needs. As a result, relevant but unexpressed needs often go unmet. Existing proactive agents attempt to address this gap either by eliciting further clarification, preserving this burden, or by extrapolating future needs from context, often leading to unnecessary or mistimed interventions. We introduce ProPer, Proactivity-driven Personalized agents, a novel two-agent architecture consisting of a Dimension Generating Agent (DGA) and a Response Generating Agent (RGA). DGA, a fine-tuned LLM agent, leverages explicit user data to generate multiple implicit dimensions (latent aspects relevant to the user’s task but not considered by the user) or knowledge gaps. These dimensions are selectively filtered using a reranker based on quality, diversity and task-relevance. RGA then balances explicit and implicit dimensions to tailor personalized responses with timely, proactive interventions. We evaluate ProPer across multiple domains using a structured, gap-aware rubric that measures coverage, initiative appropriateness, and intent alignment. Our results show that ProPer improves mean quality scores and win rates across all domains, achieving up to 84% gains in single-turn evaluation and consistent dominance in multi-turn interactions.

Paper Type: Long

Research Area: Human-AI Interaction/Cooperation and Human-Centric NLP

Research Area Keywords: Human-Centered NLP, Dialogue and Interactive Systems, Clinical and Biomedical Applications

Contribution Types: Model analysis & interpretability, NLP engineering experiment

Languages Studied: English

Submission Number: 7710