Measuring Progress in Second Language Pronunciation Learning using Automated Assessment Metrics
Abstract: A teaching strategy using repetition has been popular for second language (L2) pronunciation learning. Built upon the strategy, the effectiveness of repetition is known to be enhanced by feedback. This study investigates the effectiveness of repetition with and without feedback as pronunciation learning strategies for Chinese learners of English, utilising multiple automated pronunciation assessment metrics. The use of automatic pronunciation assessment helps avoid the subjectivity of human evaluation, which often shows weak correlations among raters, making automated methods more reliable. A novel corpus, Repetition-based Pronunciation Improvement (RPI), was collected from 50 Chinese learners divided into two groups: repetition only (RPI\_G1) and repetition with feedback (RPI\_G2). Eighteen pronunciation assessment metrics, including automatic phone recognition, self-supervised models, and Goodness of Pronunciation (GOP) were used to evaluate learner pronunciations over 12 repetitions of 7 pseudo-words. Results show RPI\_G2 demonstrated positive learning rates across most metrics, while RPI\_G1 showed negative learning rates, indicating the importance of feedback for pronunciation improvement. Analysis of the metrics revealed varying levels of consistency and correlation, with self-supervised models showing high correlation.
Paper Type: Long
Research Area: Resources and Evaluation
Research Area Keywords: L2 Pronunciation Learning, Automated Assessment Metrics, Phoneme Error Rate, Self-supervised, GOP
Contribution Types: Model analysis & interpretability, Data analysis
Languages Studied: English
Submission Number: 681
Loading