AffectMind: Proactive Knowledge Grounding with Affective Multimodal Signals for Aligned Marketing Dialogue

Xinyu Wang, Xiaomin Zhao, Zhihao Lin, Xiang Luo, Zhang Chengbiao, Jin Cheng, Yixin Wang, Yangyang Zhang, Ernie Tian, Zhiguo Tao, Yifei Kang, Xiaofei Han, Xiaotong Ding

Keywords: Multimodal Signals, Knowledge Grounding, Multi-Agent Learning

TL;DR: AffectMind is a multimodal affective agent that proactively grounds marketing dialogue in user emotions and knowledge, improving emotional consistency, persuasion, and engagement over existing methods.

Abstract: Marketing dialogue demands responses that are simultaneously emotion-aligned, knowledge-grounded, and goal-directed across extended interactions—capabilities that current large language models lack. We propose AffectMind, a multimodal affective agent that maintains and updates both factual and affective knowledge from textual, visual, and prosodic cues in real time. AffectMind links user affect with purchase intent to condition persuasion strategy selection, while a reinforcement learning loop optimizes long-horizon behavior through engagement and emotional coherence feedback. On two multimodal marketing dialogue benchmarks, AffectMind improves emotional consistency by 26%, persuasive success rate by 19%, and user engagement by 23% over competitive baselines, demonstrating the effectiveness of proactive affective grounding for commercial dialogue systems.

Submission Number: 13