Quantized Dialog - A general approach for conversational systems

R. Chulaka Gunasekara, David Nahamoo, Lazaros C. Polymenakos, David Echeverría Ciaurri, Jatin Ganhotra, Kshitij P. Fadnis

Abstract: Highlights • Quantized Dialog relies on semantic quantization and clustering of dialog utterances. • Once the proper clusters are obtained, the dialog policy can be learned efficiently. • A language model is obtained by learning transitions probabilities between clusters. • All five language models tested show very good performance (accuracy close to 100). • Quantized Dialog appealingly simple and promising option for conversational systems. Abstractr We propose Quantized Dialog, a novel approach for the development of conversational systems. The methodology relies on the semantic quantization and clustering of the dialog utterances in order to reduce the dialog interaction space, making prediction of the next utterance more tractable. The effectiveness of this method is showcased using the goal-oriented dataset of the sixth Dialog System Technology Challenge (DSTC6). We compare the performance of Quantized Dialog based on an n-gram language model for next-utterance prediction against other models that employ popular deep-learning architectures, such as multi-layer neural network classifiers, memory networks, long short-term memory recurrent neural networks and convolutional neural networks. The experimental results demonstrate the promising potential of the new quantized approach in goal-oriented dialog prediction.