CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs

Taha Aksu, Devamanyu Hazarika, Shikib Mehri, Seokhwan Kim, Dilek Hakkani-Tur, Yang Liu, Mahdi Namazifar

Submission Type: Regular Long Paper

Submission Track: Dialogue and Interactive Systems

Submission Track 2: Natural Language Generation

Keywords: Instruction Tuning, Open Domain Dialog, Controlled text generation, Task unification, Unified grounding, Compositional learning, Compositional instructions, CESAR

TL;DR: We set a unified ground for a large set of open-domain dialog tasks and automate integration of atomic tasks into more complex compositional tasks.

Abstract:

Instruction-based multitasking has played a critical role in the success of large language models (LLMs) in multi-turn dialog applications. While publicly available LLMs have shown promising performance, when exposed to complex instructions with multiple constraints, they lag against state-of-the-art models like ChatGPT. In this work, we hypothesize that the availability of large-scale complex demonstrations is crucial in bridging this gap. Focusing on dialog applications, we propose a novel framework, CESAR, that unifies a large number of dialog tasks in the same format and allows programmatic induction of complex instructions without any manual effort.

We apply CESAR on InstructDial, a benchmark for instruction-based dialog tasks. We further enhance InstructDial with new datasets and tasks and utilize CESAR to induce complex tasks with compositional instructions. This results in a new benchmark called InstructDial++, which includes 63 datasets with 86 basic tasks and 68 composite tasks. Through rigorous experiments, we demonstrate the scalability of CESAR in providing rich instructions. Models trained on InstructDial++ can follow compositional prompts, such as prompts that ask for multiple stylistic constraints.

Submission Number: 2472