vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations

Sep 25, 2019 Blind Submission readers: everyone Show Bibtex
  • TL;DR: Learn how to quantize speech signal and apply algorithms requiring discrete inputs to audio data such as BERT.
  • Abstract: We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations. Discretization enables the direct application of algorithms from the NLP community which require discrete inputs. Experiments show that BERT pre-training achieves a new state of the art on TIMIT phoneme classification and WSJ speech recognition.
  • Keywords: speech recognition, speech representation learning
  • Original Pdf:  pdf
0 Replies