Modeling Perspective-Dependent Ambiguity in Collaborative Dialogue

Justin T Chiu, Wenting Zhao, Daniel Fried, Alexander M Rush

Abstract: Errors in reference generation and resolution can occur if a dialogue agent reasons incorrectly about their partner's perspective. We present, in a collaborative and visually grounded setting, a dialogue planner that infers its partner's perspective to produce referring expressions that the partner resolves correctly. The dialogue planner models the partner perspective as a latent variable, embedded in a partner model that is used for both model-based planning and incorporating evidence from partner responses. We validate our approach on \textsc{OneCommon}, a challenging dialogue game where players have large differences in their perspectives. In symbolic selfplay, where agents partner with a copy of themselves using symbolic communication, the dialogue planner improves over a planning baseline that does not reason about the partner's full perspective.

Archival: Non-Archival