Sample Efficiency Matters: Training Multimodal Conversational Recommendation Systems in a Low Resource Setting

Anonymous

Abstract: Multi-modal conversational recommendation (multi-modal CRS) can potentially revolutionize how customers interact with e-commerce platforms. Yet conversational samples, as training data for such a system, are difficult to obtain in large quantities, particularly in new platforms. Motivated by this challenge, we consider multimodal CRS in a low resource setting. Specifically, assuming the availability of a small number of samples with dialog states, we devise an effective dialog state encoder to bridge the semantic gap between conversation and product representations for recommendation. To reduce the cost associated with dialog state annotation, a semi-supervised learning method is developed to effectively train the dialog state encoder with a smaller set of labeled conversations. In addition, we design a correlation regularisation that leverages the multi-modal knowledge in the domain database to better align textual and visual modalities. Experiments on two datasets (MMD and SIMMC) demonstrate the effectiveness of our method. Particularly, with only 5\% of the MMD training set, our method (namely SeMANTIC) is comparable to the state-of-the-art model trained on the full dataset.

Paper Type: long

Research Area: Dialogue and Interactive Systems

Contribution Types: Approaches to low-resource settings

Languages Studied: English