TranscRater: a Tool for Automatic Speech Recognition Quality Estimation

Shahab Jalalvand, Matteo Negri, Marco Turchi, José G. C. de Souza, Daniele Falavigna, Mohammed R. H. Qwaider

Abstract: We present TranscRater, an open-source tool for automatic speech recognition (ASR) quality estimation (QE). The tool allows users to perform ASR evaluation bypassing the need of reference transcripts and confidence information, which is common to current assessment protocols. TranscRater includes: i) methods to extract a variety of quality indicators from (signal, transcription) pairs and ii) machine learning algorithms which make possible to build ASR QE models exploiting the extracted features. Confirming the positive results of previous evaluations, new experiments with TranscRater indicate its effectiveness both in WER prediction and transcription ranking tasks.