A STREAMING ON-DEVICE END-TO-END MODEL SURPASSING SERVER-SIDE CONVENTIONAL MODEL QUALITY AND LATENCY

Tara N. Sainath, Yanzhang He, Bo Li, Arun Narayanan, Ruoming Pang, Antoine Bruguier, Shuo-yiin Chang, Wei Li, Raziel Alvarez, Zhifeng Chen, Chung-Cheng Chiu, David Garcia, Alex Gruenstein, Ke Hu, Minho Jin, Anjuli Kannan, Qiao Liang, Ian McGraw, Cal Peyser, Rohit Prabhavalkar et al. (9 additional authors not shown)

Abstract:

Thus far, end-to-end (E2E) models have not been shown to outperform
state-of-the-art conventional models with respect to both quality, i.e.,
word error rate (WER), and latency, i.e., the time the hypothesis is
finalized after the user stops speaking. In this paper, we develop a
first-pass Recurrent Neural Network Transducer (RNN-T) model and
a second-pass Listen, Attend, Spell (LAS) rescorer that surpasses a
conventional model in both quality and latency. On the quality side,
we incorporate a large number of utterances across varied domains
[1] to increase acoustic diversity and the vocabulary seen by the
model. We also train with accented English speech to make the
model more robust to different pronunciations. In addition, given the
increased amount of training data, we explore a varied learning rate
schedule. On the latency front, we explore using the end-of-sentence
decision emitted by the RNN-T model to close the microphone, and
also introduce various optimizations to improve the speed of LAS
rescoring. Overall, we find that RNN-T+LAS offers a better WER and
latency tradeoff compared to a conventional model. For example, for
the same latency, RNN-T+LAS obtains a 8% relative improvement
in WER, while being more than 400-times smaller in model size.