Sommelier: Scalable Open Multi-turn Audio Pre-processing for Full-duplex Speech Language Models

Kyudan Jung, Jihwan Kim, Soyoon Kim, Jeonghoon Kim, Jaegul Choo, Cheonbok Park

Keywords: Full-duplex, Speech Language Model, Data Preprocessing

TL;DR: We propose a scalable dataset preprocessing pipeline for training full-duplex Speech Language Models.

Abstract:

As the paradigm of AI shifts from text-based LLMs to Speech Language Models (SLMs), there is a growing demand for full-duplex systems capable of real-time, natural human-computer interaction.
However, the development of such models is constrained by the scarcity of high-quality, multi-speaker conversational data, as existing large-scale resources are predominantly single-speaker or limited in volume.
Addressing the complex dynamics of natural dialogue, such as overlapping and back-channeling remains a challenge, with standard processing pipelines suffering from diarization errors and ASR hallucinations.
To bridge this gap, we present a robust and scalable open-source data processing pipeline designed for full-duplex model.
Our code and project page are publicly available at https://anonymous-2001-j.github.io/sommelier.github.io/.

Submission Type: Emerging

Submission Number: 51