Investigation of a Single-Channel Frequency-Domain Speech Enhancement Network to Improve End-to-End Bengali Automatic Speech Recognition Under Unseen Noisy Conditions

Md Mahbub E. Noor, Yen-Ju Lu, Syu-Siang Wang, Supratip Ghose, Chia-Yu Chang, Ryandhimas E. Zezario, Shafique Ahmed, Wei-Ho Chung, Yu Tsao, Hsin-Min Wang

Abstract: Due to the presence of distortion, most of the single-channel frequency-domain speech enhancement (SE) approaches are still challenging for downstream automatic speech recognition (ASR) tasks, even with satisfactory improvements in enhancing speech quality and intelligibility. Recently, transformer-based models have shown better performance in speech processing tasks. Therefore, we intend to explore a transformer-based SE model, which is fine-tuned through a two-stage training scheme. Pre-training is performed using a feature-level optimization criterion through SE loss, and then a pre-trained end-to-end ASR model is used to fine-tune the SE model using an ASR-oriented optimization criterion through SE and ASR losses. We evaluate the proposed approach on low-resourced Bengali language, which has not received as much attention as resource-rich English or Mandarin languages in both SE and ASR fields. Experimental results show that it can improve the performance of SE and ASR under severe unseen noisy conditions and its performance is reasonably good compared with other state-of-the-art SE methods.