Evaluating Automatic Speech Recognition for L2 Pronunciation Feedback: A Focus on Google Translate

This study examines the L2 pronunciation feedback provided by the Automatic Speech Recognition (ASR) functionality in Google Translate (GT). We focus on three Quebec Francophone (QF) errors in English: th-substitution, h-deletion, and h-epenthesis. Four hundred and eighty male and female QF recordings of sentences with correctly and incorrectly pronounced final items (e.g. "I don't know who to thank versus tank") were played into GT. Errors were equally divided between mispronunciations leading to real word ("thank" [right arrow] "tank") and nonword output ("thief" [right arrow] "tief"). As anticipated, we found greater transcription accuracy for correct pronunciations and, among incorrect pronunciations, for real words versus nonwords. Overall, our findings suggest ASR can be highly effective for pronunciation feedback. We also examined transcriptions for gender bias, since ASR systems are often trained on corpora with more male voices, but our concerns proved unfounded: surprisingly, higher transcription accuracy was found for female recordings. [For the complete volume, "Intelligent CALL, Granular Systems and Learner Data: Short Papers from EUROCALL 2022 (30th, Reykjavik, Iceland, August 17-19, 2022)," see ED624779.]