Does Listener Gaze in Face-to-Face Interaction Follow the Entropy Rate Constancy Principle: An Empirical Study
Submission Type: Regular Short Paper
Submission Track: Speech and Multimodality
Submission Track 2: Linguistic Theories, Cognitive Modeling, and Psycholinguistics
Keywords: entropy rate constancy principle, information density, dialogue, nonverbal behaviour, gaze
TL;DR: We investigate whether the entropy rate constancy principle also holds for listener's nonverbal behaviour (specifically gaze) in dialogue.
Abstract: It is generally assumed that language (written and spoken) follows the entropy rate constancy (ERC) principle, which states that the information density of a text is constant over time. Recently, this has also been found for nonverbal gestures used in monologue, but it is still unclear whether the ERC principle also applies to listeners' nonverbal signals. We focus on listeners' gaze behaviour extracted from video-recorded conversations and trained a transformer-based neural sequence model to process the gaze data of the dialogues and compute its information density. We also compute the information density of the corresponding speech using a pre-trained language model. Our results show (1) that listeners' gaze behaviour in dialogues roughly follows the ERC principle, as well as (2) a congruence between information density of speech and listeners' gaze behaviour.
Submission Number: 3178
Loading