Bridging Disfluent to Fluent in Speech Translation: Effective Tagging and Fine-Tuning Strategies

Yuka Ko, Katsuhito Sudoh, Satoshi Nakamura, Sakriani Sakti

Abstract: Speech translation (ST) converts speech into text or speech in the target language. A major challenge in ST is handling spontaneous speech, which often includes disfluencies such as fillers and hesitations. Fluent translations enhance readability, clarity, and usability, making disfluent-to-fluent (D2F) ST highly desirable. Generally, fine-tuning with parallel data in ST is effective, but for D2F ST, limited training data constrains performance. To mitigate the data scarcity issue in D2F ST, we explore training strategies for a disfluency-aware ST model, utilizing augmented data with disfluency tagging and multi-stage fine-tuning. Our experiments show that leveraging disfluency tagging and multi-stage fine-tuning significantly improves performance while reducing disfluencies in translation.

External IDs:dblp:conf/ococosda/KoSNS25