Consistency and reliability of voice quality ratings for different types of speech fragments

This study describes a perception experiment in which listeners were asked to rate voice fragments obtained from a variety of speakers on grade, breathiness, and roughness. Four different types of stimuli were presented to each listener. One type of stimulus was based on connected speech fragments; the other three were based on different segments of a sustained vowel, yielding a 200 msec vowel onset stimulus, a 200 msec post-onset stimulus, and a 1000 msec whole vowel stimulus. Analyses focused on the consistency and reliability of grade, roughness, and breathiness ratings. Results indicated that stimulus type had virtually no effect on either within- or between-listener consistency of the grade, breathiness, or roughness ratings. Rating reliability too was hardly influenced by stimulus type. When determined as a function of the overall degree of deviance of a voice, the reliability of breathiness and roughness ratings was slightly higher for whole vowel and vowel onset stimuli than for connected speech and post-onset stimuli. It is concluded that connected speech stimuli are not necessarily to be preferred over vowel-type stimuli for a perceptual evaluation of grade, roughness, or breathiness. The somewhat higher reliability of ratings on vowel onset and whole vowel stimuli as compared to the post-onset stimuli is taken as an indication that the onset part of a vowel may contain voice quality cues that are less salient in the most stable part of a vowel. KEY WORDS: voice quality, perceptual evaluation, consistency, reliability, types of speech fragments