Leveraging audible and inaudible signals for pronunciation training by sensing articulation through a smartphone

Aslan B. Wong, Ziqi Huang, Kaishun Wu

Abstract: Highlights•Under pronunciation principles, this paper presents a smartphone-based pronunciation training system for practicing monophthong regarding vowel articulation.•Speech and ultrasonic signals are captured and analyzed simultaneously to determine the tongue position and shape of the lips respecting uttering vowels.•The proposed system implementing a commercial smartphone is robust with high accuracy for practical scenarios.•The evaluation for its impact on user experience shows that the proposed system has raised learning engagement and enjoyment.