Accuracy of Speech Sound Analysis: Comparison of an Automatic Artificial Intelligence Algorithm with Clinician Assessment

Purpose: Automatic speech analysis (ASA) and automatic speech recognition systems are increasingly being used in the treatment of speech sound disorders (SSDs). When utilized as a home practice tool or in the absence of the clinician, the ASA system has the potential to facilitate treatment gains. However, the feedback accuracy of such systems varies, a factor that may impact these gains. The current research analyzes the feedback accuracy of a novel ASA algorithm (Amplio Learning Technologies), in comparison to clinician judgments. Method: A total of 3,584 consonant stimuli, produced by 395 American English-speaking children and adolescents with SSDs (age range: 4-18 years), were analyzed with respect to automatic classification of the ASA algorithm, clinician-ASA agreement, and interclinician agreement. Further analysis of results as related to phoneme acquisition categories (early-, middle-, and late-acquired phonemes) was conducted. Results: Agreement between clinicians and ASA classification for sounds produced accurately was above 80% for all phonemes, with some variation based on phoneme acquisition category (early, middle, late). This variation was also noted for ASA classification into "acceptable," "unacceptable," and "unknown" (which means no determination of phoneme accuracy) categories, as well as interclinician agreement. Clinician-ASA agreement was reduced for misarticulated sounds. Conclusions: The initial findings of Amplio's novel algorithm are promising for its potential use within the context of home practice, as it demonstrates high feedback accuracy for correctly produced sounds. Furthermore, complexity of sound influences consistency of perception, both by clinicians and by automated platforms, indicating variable performance of the ASA algorithm across phonemes. Taken together, the ASA algorithm may be effective in facilitating speech sound practice for children with SSDs, even in the absence of the clinician.