Sentence Retrieval for Open-Ended Dialogue Using Dual Contextual Modeling

Hagai Taitelbaum, Itay Harel, Idan Szpektor, Oren Kurland

Abstract: We address the task of retrieving sentences for an open domain dialogue that contain information useful for generating the next turn. We propose several novel neural retrieval architectures based on dual contextual modeling: the dialogue context and the context of the sentence in its ambient document. The architectures utilize contextualized language models (BERT), fine-tuned on a large-scale dataset constructed from Reddit. We evaluate the models using a recently published dataset. The performance of our most effective model is substantially superior to that of strong baselines.