An Efficient Joint Training Framework for Robust Small-Footprint Keyword Spotting

Yue Gu, Zhihao Du, Hui Zhang, Xueliang Zhang

Abstract: In real-world applications, robustness against noise is crucial for small-footprint keyword spotting (KWS) systems which are deployed on resource-limited devices. To improve the noise robustness, a reasonable approach is employing a speech enhancement model to enhance the noisy speeches first. However, current enhancement models need a lot of parameters and computation, which do not satisfy the small-footprint requirement. In this paper, we design a lightweight enhancement model, which consists of the convolutional layers for feature extracting, recurrent layers for temporal modeling and deconvolutional layers for feature recovering. To reduce the mismatch between the enhanced features and KWS system desired ones, we further propose an efficient joint training framework, in which the enhancement model and KWS system are concatenated and jointly fine-tuned through a trainable feature transformation block. With the joint training, linguistic information can back-propagate from the KWS system to the enhancement model and guide its training. Our experimental results show that the proposed small-footprint enhancement model significantly improves the noise robustness of KWS systems without much increasing model or computation complexity. Moreover, the recognition performance can be further improved through the proposed joint training framework.