Understanding Emotion in Discourse: Recognition Insights and Linguistic Patterns for Generation

TMLR Paper6840 Authors

Abstract:

Despite strong recent progress in Emotion Recognition in Conversation (ERC), two gaps remain: we still lack a clear understanding of which modeling choices materially affect performance, and we have limited linguistic analysis that links recognition findings to actionable cues for generation. We address both gaps via a systematic study on IEMOCAP.

For recognition, we conduct controlled ablations with 10 random seeds and paired tests over seeds (with correction for multiple comparisons), yielding three findings. First, conversational context is the dominant factor: performance saturates quickly, with roughly 90% of the gain observed within our context sweep achieved using only the most recent 10--30 preceding turns (depending on the label set). Second, hierarchical sentence representations improve utterance-only recognition ($K{=}0$), but the benefit vanishes once turn-level context is available, suggesting that conversational history subsumes much of the intra-utterance structure. Third, a simple integration of an external affective lexicon (SenticNet) does not improve results, consistent with pretrained encoders already capturing much of the affective signal needed for ERC. Under a strictly causal (past-only) setting, our simple models attain strong performance (82.69% 4-way; 67.07% 6-way weighted F1), indicating that competitive accuracy is achievable without access to future turns.

For linguistic analysis, we examine 5,286 discourse-marker occurrences and find a reliable association between emotion and marker position within the utterance ($p < 0.0001$). In particular, "Sad" utterances show reduced left-periphery marker usage (21.9%) relative to other emotions (28--32%), aligning with accounts that link left-periphery markers to active discourse management. This pattern is consistent with our recognition results, where "Sad" benefits most from conversational context (+22%p), suggesting that sadness often relies more on discourse history than on overt pragmatic signaling in the utterance itself.

Submission Type: Regular submission (no more than 12 pages of main content)

Assigned Action Editor: ~Ali_Etemad1

Submission Number: 6840