Search

Purpose: Lipreading and its cognitive correlates were studied in school-age children with typical language development and delayed language development due to specific language impairment (SLI). Method: Forty-two children with typical language development and 20 children with SLI were tested by using a word-level lipreading test and an extensive battery of standardized cognitive and linguistic tests. Results: Children with SLI were poorer lipreaders than their typically developing peers. Good phonological skills were associated with skilled lipreading in both typically developing children and in children with SLI. Lipreading was also found to correlate with several cognitive skills, for example, short-term memory capacity and verbal motor skills. Conclusions: Speech processing deficits in SLI extend also to the perception of visual speech. Lipreading performance was associated with phonological skills. Poor lipreading in children with SLI may be, thus, related to problems in phonological processing., Visual information from a speaker's face plays an important role in understanding spoken language. In noisy conditions, watching a speaker's facial movements improves speech perception (MacLeod & Summerfield, 1987; Sumby [...]

Purpose: Prior studies (e.g., Nath & Beauchamp, 2012) report large individual variability in the extent to which participants are susceptible to the McGurk effect, a prominent audiovisual (AV) speech illusion. The current study evaluated whether susceptibility to the McGurk effect (MGS) is related to lipreading skill and whether multiple measures of MGS that have been used previously are correlated. In addition, it evaluated the test-retest reliability of individual differences in MGS. Method: Seventy-three college-age participants completed 2 tasks measuring MGS and 3 measures of lipreading skill. Fifty-eight participants returned for a 2nd session (approximately 2 months later) in which MGS was tested again. Results: The current study demonstrated that MGS shows high test-retest reliability and is correlated with some measures of lipreading skill. In addition, susceptibility measures derived from identification tasks were moderately related to the ability to detect instances of AV incongruity. Conclusions: Although MGS is often cited as a demonstration of AV integration, the results suggest that perceiving the illusion depends in part on individual differences in lipreading skill and detecting AV incongruity. Therefore, individual differences in susceptibility to the illusion are not solely attributable to individual differences in AV integration ability., Speech perception is not an exclusively auditory process. Instead, visual information about a speaker, such as mouth shape and facial movements, supplement the auditory signal (Sumby & Pollack, 1954). As [...]

Purpose: The study addressed three research questions: Does lipreading improve between the ages of 7 and 14 years? Does hearing loss affect the development of lipreading? How do individual differences in lipreading relate to other abilities? Method: Forty children with normal hearing (NH) and 24 with hearing loss (HL) were tested using 4 lipreading instruments plus measures of perceptual, cognitive, and linguistic abilities. Results: For both groups, lipreading performance improved with age on all 4 measures of lipreading, with the HL group performing better than the NH group. Scores from the 4 measures loaded strongly on a single principal component. Only age, hearing status, and visuospatial working memory were significant predictors of lipreading performance. Conclusions: Results showed that children's lipreading ability is not fixed but rather improves between 7 and 14 years of age. The finding that children with HL lipread better than those with NH suggests experience plays an important role in the development of this ability. In addition to age and hearing status, visuospatial working memory predicts lipreading performance in children, just as it does in adults. Future research on the developmental time-course of lipreading could permit interventions and pedagogies to be targeted at periods in which improvement is most likely to occur. Key words: lipreading, children, deafness, development, The ability to encode and use visual speech information is often critical to effective spoken communication. This ability is obviously of particular importance for individuals with hearing loss (HL). In [...]

Purpose: In this article, the authors describe the development of a new instrument, the Test of Child Speechreading (ToCS), which was specifically designed for use with deaf and hearing children. Speechreading is a skill that is required for deaf children to access the language of the hearing community. ToCS is a deaf-friendly, computer-based test that measures child speechreading (silent lipreading) at 3 psycholinguistic levels: (a) Words, (b) Sentences, and (c) Short Stories. The aims of the study were to standardize the ToCS with deaf and hearing children and to investigate the effects of hearing status, age, and linguistic complexity on speechreading ability. Method: Eighty-six severely and profoundly deaf children and 91 hearing children participated. All children were between the ages of 5 and 14 years. The deaf children were from a range of language and communication backgrounds, and their preferred mode of communication varied. Results: Speechreading skills significantly improved with age for both groups of children. There was no effect of hearing status on speechreading ability, and children from both groups showed similar performance across all subtests of the ToCS. Conclusion: The ToCS is a valid and reliable assessment of speechreading ability in school-age children that can be used to measure individual differences in performance in speechreading ability. Key Words: deafness, speech perception, children, development, lipreading, speechreading, Typical face-to-face communication is multimodal, and speech perception involves the integration of both auditory and visual information (Rosenblum, 2005). The integration of visual and auditory speech seems to occur very [...]

Problem statement: Deaf and dumb needs assistance from a technical box that takes movements of lips to identify the words. This technical article provided appropriate model implementation of flexible lip model for better visual lip reading system. Approach: From the frame sequence of words, Active Shape Model (ASM) based lip model provided local tracking and extraction of geometric lip-feature. Two geometric criteria define required geometric features and its variations in the sequence. Results: The feature established machine classification using Analytic Hierarchy Process (AHP), a relative weight finder. AHP presents weight vector to fuzzy classifier to decide the video frame sequence belonging to a respective word. Conclusion: The suggested model tested on a total of 5 different sample databases results in 83.2% accuracy over the other combinational algorithms. Key words: Active shape model, analytic hierarchy process, fuzzy classifier, INTRODUCTION Identification of a word is from a video frame-sequence with the help of a statistical classifier using a snapshot database has been attempted in this study. Compared to the [...]

Listeners hearing an ambiguous phoneme flexibly adjust their phonetic categories in accordance with information telling what the phoneme should be (i.e., recalibration). Here the authors compared recalibration induced by lipread versus lexical information. Listeners were exposed to an ambiguous phoneme halfway between/t/and /p/ dubbed onto a face articulating /t/ or /p/ or embedded in a Dutch word ending in /t/ (e.g., groot [big]) or /p/ (knoop [button]). In a posttest, participants then categorized auditory tokens as /t/ or /p/. Lipread and lexical aftereffects were comparable in size (Experiment 1), dissipated about equally fast (Experiment 2), were enhanced by exposure to a contrast phoneme (Experiment 3), and were not affected by a 3-min silence interval (Experiment 4). Exposing participants to 1 instead of both phoneme categories did not make the phenomenon more robust (Experiment 5). Despite the difference in nature (bottom-up vs. top-down information), lipread and lexical information thus appear to serve a similar role in phonetic adjustments. Keywords: recalibration, speech perception, perceptual learning, lipreading, lexical information

Purpose: To study the role of visual perception of phonemes in visual perception of sentences and words among normal-hearing individuals. Method: Twenty-four normal-hearing adults identified consonants, words, and sentences, spoken by either a human or a synthetic talker. The synthetic talker was programmed with identical parameters within phoneme groups, hypothetically resulting in simplified articulation. Proportions of correctly identified phonemes per participant, condition, and task, as well as sensitivity to single consonants and clusters of consonants, were measured. Groups of mutually exclusive consonants were used for sensitivity analyses and hierarchical cluster analyses. Results: Consonant identification performance did not differ as a function of talker, nor did average sensitivity to single consonants. The bilabial and labiodental clusters were most readily identified and cohesive for both talkers. Word and sentence identification was better for the human talker than the synthetic talker. The participants were more sensitive to the clusters of the least visible consonants with the human talker than with the synthetic talker. Conclusions: It is suggested that ability to distiguish between clusters of the least visually distinct phonemes is important in speechreading. Specifically, it reduces the number of candidates, and thereby facilitates lexical identification. KEY WORDS: speechreading, articulation, students, normal hearing, For normal-hearing persons under normal listening conditions, speech perception and understanding are effortless and accurate. If the acoustic or sensory information is degraded (e.g., by noise or hearing impairment) seeing [...]

A program designed to evaluate the benefits of an audiovisual-frequency modulated (FM) system led to some questions concerning the effects of illumination level and a talker's skin color on speech-reading performance. To address those issues, the speech of a Caucasian female was videotaped under 2 conditions: a light skin color condition and a dark skin color condition. For the latter condition, makeup was applied to the talker's face. For both skin color conditions, the talker was recorded while speaking sentences under 7 different levels of illumination: 2, 3, 4, 16, 60, 256, and 600 footcandles (fc). Fifteen participants completed the speech perception task in a visual-only modality. The results revealed a significant interaction of illumination level and skin color. For the light skin color condition, speech-reading performance improved systematically as the illumination level increased from 3 to 16 fc. For the dark skin color condition, no differences in speech-reading performance were observed between the 2-fc and the 3-fc conditions. However, a large improvement in speech-reading performance was observed as the illumination level increased from 4 fc to 16 fc. It is speculated that in addition to an overall effect of illumination level, the contrast in luminance at the level of the talker's face has an effect on speech-reading performance. KEY WORDS: speech reading, lipreading, audiovisual-FM system, illumination level, skin color, Understanding the teacher's speech constitutes one of the major challenges for students who have a hearing loss. Even when they use appropriately fitted amplification systems (such as hearing aids, cochlear [...]

The immediate serial recall of lipread material is disrupted by irrelevant speech (Campbell & Dodd, 1984; Jones, 1994) and by articulatory suppression (Campbell & Dodd, 1984). However, the interaction between these has not been directly examined. In Exp 1, participants recalled silently presented lipread digits in conditions of quiet, irrelevant speech, articulatory suppression and suppression/speech combined. Irrelevant speech disrupted recall, but not when articulation was suppressed. Experiment 2 demonstrated that participants were able to accurately lipread targets in all of the above experimental conditions. A third experiment contrasted predictions derived from the phonological loop model (PL; Baddeley, 1986) and the changing state hypothesis (CSH; Jones, 1993). The CSH predicts that tones and speech that vary in frequency to the same degree will disrupt recall equally (Jones & Macken, 1993), whereas the PL model implies that speech will always be more disruptive. The results support the CSH, and extend the findings of Jones and Macken (1993) to account for lipread stimuli. As with graphic presentation, the CSH provides a better account of the processes underpinning the irrelevant speech effect; however, it is argued that the recoding hypothesis from the PL model should be retained., W. Divin (*) The irrelevant speech effect A well-established finding is that irrelevant speech disrupts the serial recall of visually presented target items (e.g. Colle & Welsh, 1976; Jones, Madden, [...]

Using digital video technology, selective aspects of a face can be masked by identifying the pixels that represent it and then, by adjusting the gray levels, effectively eliminate that facial aspect. In groups of young adults with normal vision and hearing, consonant-viseme recognition was measured for closed sets of vowel-consonant-vowel disyllables. In the first experiment viseme recognition was measured while the tongue and teeth were masked and while the entire mouth was masked. The results showed that masking of the tongue and teeth had little effect on viseme recognition, and when the entire mouth was masked, participants continued to identify consonant visemes with 70% or greater accuracy in the /a/ and /o/ vowel contexts. In the second experiment, viseme recognition was measured when the upper part of the face and the mouth were masked and when the lower part of the face and the mouth were masked. The results showed that when the mouth and the upper part of the face were masked, performance was poor, but information was available to identify the consonant-viseme /f/. When the mouth and the lower part of the face were masked, viseme recognition was quite poor, but information was available to discriminate the consonant-viseme /p/ from other consonant visemes. KEY WORDS: speechreading, lipreading, visual masking, viseme recognition

It is well-known that facial orientation affects the processing of static facial information, but similar effects on the processing of visual speech have yet to be explored fully. Three experiments are reported in which the effects of facial orientation on visual speech processing were examined using a talking face presented at 8 orientations through 360[degrees]. Auditory and visual forms of the syllables /ba/, /bi/, /ga/, /gi/, /ma/, /mi/, /ta/, and /ti/ were used to produce the following speech stimulus types: auditory, visual, congruent audiovisual, and incongruent audiovisual. Facial orientation did not affect identification of visual speed per se or the near-perfect accuracy of auditory speech report with congruent audiovisual speech stimuli. However, facial orientation did affect the accuracy of auditory speech report with incongruent audiovisual speech stimuli. Moreover, the nature of this effect depended on the type of incongruent visual speech used. Implications for the processing of visual and audiovisual speech are discussed.

Seeing a talker's face can improve the perception of speech in noise. There is little known about which characteristics of the face are useful for enhancing the degraded signal. In this study, a point-light technique was employed to help isolate the salient kinematic aspects of a visible articulating face. In this technique, fluorescent dots were arranged on the lips, teeth, tongue, cheeks, and jaw of an actor. The actor was videotaped speaking in the dark, so that when shown to observers, only the moving dots were seen. To test whether these reduced images could contribute to the perception of degraded speech, noise-embedded sentences were dubbed with the point-light images at various signal-to-noise ratios. It was found that these images could significantly improve comprehension for adults with normal hearing and that the images became more effective as participants gained experience with the stimuli. These results have implications for uncovering salient visual speech information as well as in the development of telecommunication systems for listeners who are hearing impaired. KEY WORDS: speechreading, lipreading, point-lights, telecommunication, speech in noise

The benefits of two tactile codes of voice fundamental frequency ([F.sub.0]) were evaluated as supplements to the speechreading of sentences in two short-term training studies, each using 12 adults with normal hearing. In Experiment 1, a multichannel spatiotemporal display of [F.sub.0], known as Portapitch, was used to stimulate the index finger. In an attempt to improve on past performance with this display, the coding scheme was modified to better cover the [F.sub.0] range of the talker in the training materials. For Experiment 2, to engage kinesthetic/proprioceptive pathways, a novel single-channel positional display was built, in which [F.sub.0] was coded as the vertical displacement of a small finger-rest. Input to both displays consisted of synthesized replicas of the [F.sub.0] contours of the sentences, prepared and perfected off-line. Training with the two tactile [F.sub.0] displays included auditory presentation of the synthesized [F.sub.0] contours in conjunction with the tactile patterns on alternate trials. Speechreading enhancement by the two tactile [F.sub.0] displays was compared to the enhancement provided when auditory [F.sub.0] information was available in conjunction with the tactile patterns, by auditory presentation of a sinusoidal indication of the presence or absence of voicing, and by a single-channel tactile display of the speech waveform presented to the index finger. Despite the modified coding strategy, the multichannel Portapitch provided a mean tactile speechreading enhancement of 7 percentage points, which was no greater than that found in previous studies. The novel positional [F.sub.0] display provided only a 4 percentage point enhancement. Neither [F.sub.0] display was better than the simple single-channel tactile transform of the full speech waveform, which gave a 7 percentage point enhancement effect. Auditory speechreading enhancement effects were 17 percentage points with the voicing indicator and approximately 35 percentage points when the auditory [F.sub.0] contour was provided in conjunction with the tactile displays. The findings are consistent with the hypothesis that subjects were not taking full advantage of the [F.sub.0] variation information available in the outputs of the two experimental tactile displays. KEY WORDS: speechreading, lipreading, tactile aids, speech perception, fundamental frequency

The benefit derived from visual cues in auditory-visual speech recognition and patterns of auditory and visual consonant confusions were compared for 20 middle-aged and 20 elderly men who were moderately to severely hearing impaired. Consonant-vowel nonsense syllables and CID sentences were presented to the subjects under auditory-only, visual-only, and auditory-visual test conditions. Benefit was defined as the difference between the scores in the auditory-only and auditory-visual conditions. The results revealed that the middle-aged and elderly subjects obtained similar benefit from visual cues in auditory-visual speech recognition. Further, patterns of consonant confusions were similar for the two groups.

An experiment was conducted to examine the processes involved in lipreading as well as to investigate an optimal approach to teaching lipreading skill. We compared discovery and expository methods of learning to lip-read. Twenty-six college students with normal hearing were trained over 3 days to lip-read consonant-vowel (CV) syllables. The training material consisted of a prerecorded videotape of four different talkers. The task was a forced-choice procedure with feedback. Subjects learned with training, but there was no difference between the two learning methods. As a retention measure, subjects returned 4 weeks later and repeated the training. There were significant savings of the original learning. Three weeks after the retention phase, subjects were tested with a 10-item forced-choice monosyllabic word task. Those subjects who had extensive training on CV syllables did no better on identifying the monosyllabic words than did a control group of 9 subjects with no training. Nevertheless, performance for all three groups (discovery, expository, and no training) improved during training in the word identification task.

Recognition of words in sentences of known topic was measured in normally hearing adults via speechreading alone and speechreading supplemented with auditory presentation of signals intended to convey variations of voice fundamental frequency (F(sub o)) over time. Three signals were used: (a) the low-pass filtered output of an electroglottograph (unprocessed F(sub o)), (b) a constant amplitude sine wave whose instantaneous frequency was intended to equal that of F(sub o) (processed F(sub o)), and (c) the same sine wave restricted to a small number of discrete frequency steps (quantized F(sub o)). As the number of steps in the quantized F(sub o) contours increased from 1 to 12, the speechreading enhancement effect increased. The quantized F(sub o) contour with 12 steps was as effective as the processed F(sub o) contour (without quantization), but this processed contour was significantly less effective than the unprocessed electroglottograph signal. The results show that the auditory F(sub o) speechreading enhancement effect is sensitive to the errors introduced by the F(sub o) extraction and regeneration process used in this study. It is also sensitive to the quantization of F(sub o) contours into less than 12 steps. Whether more than 12 steps are required for the full enhancement effect remains to be determined.

Generalizability theory (Cronbach, Gleser, Nanda, & Rajaratnam, 1972) was used to estimate the percentage of variance explained by three sources of variability in speechreading sentences: the subject, the talker, and the sentence materials. Videodisc recordings of the 100 CID Everyday Sentences (Davis & Silverman, 1970), spoken by a male and a female talker, were presented to 104 subjects with normal hearing. For performance on individual sentences (total number of words correct), the most important systematic sources of variability were the sentence (26.3%), the speechreader (10.5%), the talker (4.9%), and the interaction of talker and sentence (5.1%). Residual error accounted for 51.2% of the variance. Generalizability functions are presented, as a function of test length, for five models of test administration and interpretation. For 10-, 50-, and 100-item lists, generalizability is predicted to be .70,.92, and .96, respectively for a single talker. Psychometric characteristics of these recordings of the CID sentences are also presented.

Two hundred and twelve members of the Self Help for Hard of Hearing People (SHHH) organization completed a survey that evaluated their use of communication strategies. Reported use of strategies was correlated with attitudinal variables and social-interaction indices. On average, subjects agreed most strongly with questionnaire items stating they would ask a talker to repeat a misperceived utterance. They agreed less strongly with items stating they would ask the talker to restructure or elaborate an utterance. Subjects agreed strongly that they would use the communication strategies with familiar talkers, and less strongly that they would use them with unfamiliar talkers. Subjects who appeared less likely to say nothing after misperceiving an utterance were more likely to disagree that they were frustrated with their speechreading skills, and they appeared less likely to avoid social interactions. Subjects who indicated a greater likelihood of using anticipatory strategies, such as reviewing potential vocabulary before an appointment, were on average more likely to avoid social interactions. They also agreed more strongly that poor speechreaders appear less intelligent.

Similarity ratings of pairs of lipread consonants were obtained using a 5-point scale. Matrices were constructed showing mean similarity ratings and confusions among stimuli. Both the similarity and the confusion data provide normative data useful for researches in many areas. Lipread data collected here are compared with similarity ratings of orthographicallyh and auditorily presented consonants collected by Manning (1977). These comparisons provide information about how stimulus similarity both within and between presentation formats may affect information processing of the three types of stimuli. These data are of special interest to researches studying the visual processing of speech and the effect of format of presentation on recall.

It is well known that the right side of the mouth moves more than the left during speech, but little is known about how this asymmetry affects lipreading. We investigated asymmetries in the visual expression and perception of speech using the McGurk effect-an illusion in which incongruent lip movements cause listeners to misreport sounds. Thirty right-handed participants watched film clips in which the left, the right, or neither side of the mouth was covered. The McGurk effect was attenuated when the right side of the mouth was covered, demonstrating that this side is more important to lipreading than is the left side of the mouth. Mirror-reversed images tested whether the asymmetry was the result of an observer bias toward the left hemispace. The McGurk effect was stronger in the normal than in the mirror orientation when the mouth was fully visible. Thus, observers attend more to what they think is the right side of the speaker's mouth. Asymmetries in mouth movements may reflect the gestural origins of language, which are also right lateralized.

In manual-cued speech (MCS) a speaker produces hand gestures to resolve ambiguities among speech elements that are often confused by speechreaders. The shape of the hand distinguishes among consonants; the position of the hand relative to the face distinguishes among vowels. Experienced receivers of MCS achieve nearly perfect reception of everyday connected speech. MCS has been taught to very young deaf children and greatly facilitates language learning, communication, and general education. This manuscript describes a system that can produce a form of cued speech automatically in real time and reports on its evaluation by trained receivers of MCS. Cues are derived by a hidden markov models (HMM)-based speaker-dependent phonetic speech recognizer that uses context-dependent phone models and are presented visually by superimposing animated handshapes on the face of the talker. The benefit provided by these cues strongly depends on articulation of hand movements and on precise synchronization of the actions of the hands and the face. Using the system reported here, experienced cue receivers can recognize roughly two-thirds of the keywords in cued low-context sentences correctly, compared to roughly one-third by speechreading alone (SA). The practical significance of these improvements is to support fairly normal rates of reception of conversational speech, a task that is often difficult via SA. Index Terms--Aids for the deaf, autocuer, automatic speech recognition, cued speech, deafness, speech perception, speechreading, transliteration.

Psychological differentiation, psychophysiological arousal, hearing capacity, and ability to lipread unrelated sentences presented in a silent technicolor film were investigated in 62 children (mean age 10 years, 4 months), classified into three hearing categories (normal, mild to moderate hearing loss, severe hearing loss) based on their average hearing threshold in the range of speech frequencies for their best ear. All subjects were administered the Rod and Frame Test, Craig Lipreading Inventory, and skin conductance measures under two conditions (pre-task and task). The results showed that the ability to lipread sentences from a film was related significantly to psychological differentiation (cognitive style) and pre-task conductance level, but was not related to hearing capacity or task conductance level. The best predictors of lipreading ability were successful performance on the Rod and Frame Test and high increases in skin conductance from pre-task to task situation.

Two tests of lipreading ability were constructed, one using numbers and the other of sentences including visual cues. The tests were given to two groups of men, one older group aged 40 yr. and over (n = 110) and a younger group of less than 40 yr. (n = 70). Requests to guess produced a higher mean score for the older subjects on the lipreading tests containing the greater amount of information. It is suggested that differences in the effect of encouragement on performance between young and old may be related to both age and cultural factors.

Ninety-six participants with normal hearing and 63 with severe-to-profound hearing impairment viewed 100 CID Sentences (Davis & Silverman, 1970) and 100 B-E Sentences (Bernstein & Eberhardt, 1986b). Objective measures included words correct, phonemes correct, and visual-phonetic distance between the stimulus and response. Subjective ratings were made on a 7-point confidence scale. Magnitude of validity coefficients ranged from .34 to .76 across materials, measures, and groups. Participants with hearing impairment had higher levels of objective performance, higher subjective ratings, and higher validity coefficients, although there were large individual differences. Regression analyses revealed that subjective ratings are predictable from stimulus length, response length, and objective performance. The ability of speechreaders to make valid performance evaluations was interpreted in terms of contemporary word recognition models. KEY WORDS: speechreading (lipreading), word recognition, subjective ratings

Intrafacial differences in speechreading performance were studied measuring 24 subjects' visual recognition of key words within sentences when presented either a left-left-(LL) or right-right-(RR)-sided face of a male speaker with moderate facial asymmetry. No significant differences were found as a function of facial side or gender (ns = 12).