vq-wav2vec: Self-Supervised Learning of Discrete Speech RepresentationsDownload PDF

Published: 20 Dec 2019, Last Modified: 22 Oct 2023ICLR 2020 Conference Blind SubmissionReaders: Everyone
TL;DR: Learn how to quantize speech signal and apply algorithms requiring discrete inputs to audio data such as BERT.
Abstract: We propose vq-wav2vec to learn discrete representations of audio segments through a wav2vec-style self-supervised context prediction task. The algorithm uses either a gumbel softmax or online k-means clustering to quantize the dense representations. Discretization enables the direct application of algorithms from the NLP community which require discrete inputs. Experiments show that BERT pre-training achieves a new state of the art on TIMIT phoneme classification and WSJ speech recognition.
Keywords: speech recognition, speech representation learning
Code: [![Papers with Code](/images/pwc_icon.svg) 3 community implementations](https://paperswithcode.com/paper/?openreview=rylwJxrYDS)
Data: [LibriSpeech](https://paperswithcode.com/dataset/librispeech), [TIMIT](https://paperswithcode.com/dataset/timit)
Community Implementations: [![CatalyzeX](/images/catalyzex_icon.svg) 3 code implementations](https://www.catalyzex.com/paper/arxiv:1910.05453/code)
Original Pdf: pdf
10 Replies