Is this Dialogue Coherent? Learning from Dialogue Acts and Entities

Alessandra Cervone, giuseppe riccardi

Abstract: In this work, we investigate the human percep- tion of coherence in open-domain dialogues. In particular we address the problem of an- notating and modeling the coherence of next- turn candidates while considering the entire history of the dialogue. First, we create the Switchboard Coherence (SWBD-Coh) corpus, a dataset of human-human spoken dialogues annotated with turn coherence ratings, where next-turn candidate utterances ratings are pro- vided considering the full dialogue context. Our statistical analysis of the corpus indicates how turn coherence perception is affected by patterns of distribution of entities previously introduced and the Dialogue Acts used. Sec- ond, we experiment with different architec- tures to model entities, Dialogue Acts and their combination and evaluate their perfor- mance in predicting human coherence ratings on SWBD-Coh. We find that models jointly combining both DA and entity information yield the best performances both for response selection and turn coherence rating.