Many Hands Make Light Work: Task-Oriented Dialogue System with Module-Based Mixture-of-Experts

Anonymous

Abstract:

Task-oriented dialogue systems are broadly used in virtual assistants and other automated services, providing interfaces between users and machines to facilitate specific tasks.
For example, in the context of hotel reservations, these systems not only recommend hotels that align with user preferences but also retain user requirements for future reference.
Corresponding to a wide range of properties and applications of task-oriented dialogue systems, their outputs may also be diverse.
Nowadays, task-oriented dialogue systems have benefited greatly from pre-trained language models (PLMs).
While being effective and performant, scaling these models is expensive and complex.
To address these challenges, we propose SMETOD to generate diverse natural language outputs, which scales the capacity of a task-oriented dialogue system while maintaining efficient inference.
We extensively evaluate our model on dialogue state tracking, dialogue response generation, and intent prediction.
Experimental results demonstrate that SMETOD consistently achieves state-of-the-art or comparable performance on all evaluated datasets.
Furthermore, SMETOD shows an advantage in the cost of inference compared to existing approaches.

Paper Type: long

Research Area: Dialogue and Interactive Systems

Contribution Types: Model analysis & interpretability, Publicly available software and/or pre-trained models

Languages Studied: English

Consent To Share Submission Details: On behalf of all authors, we agree to the terms above to share our submission details.