Vox-Infinity: Benchmarking the Limits of Long-Context Spoken Language Models

Xize Cheng, Dongjie Fu, Chenyuhao Wen, Tao Jin, Hai Yu, Di Cao, Qinying Liu, Yexin Yang, Zehan Wang, Shengpeng Ji, Siqi Zheng, Xu Tan, Zhou Zhao

Keywords: long context modeling, audio benchmark

TL;DR: A Benchmark for Long Context Spoken Language Models.

Abstract: Long-context reasoning remains a fundamental challenge for large language models, as excessively long inputs often lead to the forgetting of salient information. This issue is even more pronounced in the speech domain, where audio, as a low-compression modality, requires significantly more embeddings than text to preserve both semantic content and acoustic cues. To address this, we introduce \textbf{Vox-Infinity}, the first benchmark specifically designed to evaluate long-context understanding in spoken language models. Vox-Infinity systematically extends audio history along two dimensions: turns and duration. It covers a diverse range of representative scenarios, including dialogues with varying structural depth and semantic complexity. Crucially, it provides explicit answer provenance annotations and organizes samples based on the context length required to resolve each query, enabling precise and length-aware evaluation of model performance. Furthermore, we present the first comprehensive study of history modeling strategies in this setting, analyzing how models balance the trade-off between preserving long-range semantics and retaining recent acoustic signals. Cases and datasets are available at \url{https://vox-infinity.github.io}.

Primary Area: datasets and benchmarks

Submission Number: 17727