Audio-visual speech recognition for a vowel discrimination task

Peter L. Silsbee, Alan Conrad Bovik

Abstract: Among the various methods which have been proposed to improve the robustness and accuracy of automatic speech recognition (ASR) systems, lipreading has received very little attention. In this paper, we provide motivation for the use of lipreading. A novel speaker dependent lipreading system is developed, which uses hidden Markov modeling, a well known and highly successful technique for audio-based ASR. It is used in conjunction with an audio ASR system in order to improve the accuracy of the latter, especially under degraded acoustical conditions. Reductions in error of 30 to over 60% result.