Search

We present a hand and face segmentation methodology using color and motion cues for the content-based representation of sign language video sequences. The methodology consists of three stages, namely skin-color segmentation, change detection, and face and hand segmentation mask generation. In skin-color segmentation, a universal color-model is derived and image pixels are classified as skin or nonskin based on their Mahalanobis distance. We derive a segmentation threshold for the classifier. The aim of change detection is to localize moving objects in a video sequences. The change detection technique is based on the F test and block-based motion estimation. Finally, the results from skin-color segmentation and change detection are analyzed to segment the face and hands. The performance of the algorithm is illustrated by simulations carried out on standard test sequences. Index Terms--Change detection, sign language, skin-color segmentation, video segmentation.

DEAF PEOPLE HAVE difficulty reading and remembering English script because of its sound-based orthography. Logographs (e.g., kanji, Arabic numerals) should not pose the same challenge because they are based on meaning, not sound. Little research has been conducted to test this theory's validity cross-culturally. The present study was an attempt to do just that. The first of two experiments tested immediate memory spans for word sequences of 20 hearing Irish, 20 prelingually deaf Americans, 20 hearing Japanese, and 20 prelingually deaf Japanese. For English words, deaf participants showed shorter memory spans than hearing participants, but memory spans were similar for deaf and hearing participants for words in kanji, the logographic system for Japanese writing. The second experiment tested memory span for Arabic numerals, with the same participants. Deaf English-readers showed shorter memory spans than their hearing counterparts, but deaf and hearing Japanese performed similarly.

ACCORDING TO ENROLLMENTS in schools for the deaf and data from the national census and neonatal hearing screening programs, the incidence of severe and profound childhood deafness in Australia is, and has been, less than commonly assumed. Factors implicated include improved medical care, mainstreaming, cochlear implants, and genetic science. Data for the United States, Britain, and other developed countries seem consistent with those for Australia. Declining prevalence and incidence rates have immediate implications for sign-based education, teacher-of-the-deaf training programs, and educational interpreting. There are also serious consequences for research, documentation, and teaching regarding Australian Sign Language (Auslan), and for the future viability of Auslan. Prompt action is essential if a credible corpus of Auslan is to be collected as the basis for a valid and verifiable description of one of the few native sign languages in the world with significant attested historical depth.

Long-term use of a sign language may enhance visual abilities. Deaf signers consistently outscore their hearing nonsigning counterparts on visual memory tests. In a similar vein, the long-term use of a logographic script, Chinese characters appears to boost visual memory. Both learning a sign language and mastering Chinese characters involve a relatively complex task of visual recognition and memory. This review of the literature suggests that daily use of either a sign language, one not based on sound, or Chinese characters, a script which maps language at the word level rather than at the phonemic level, may facilitate recall of visual forms.

We present an algorithm for extracting and classifying two-dimensional motion in an image sequence based on motion trajectories. First, a multiscale segmentation is performed to generate homogeneous regions in each frame. Regions between consecutive frames are then matched to obtain two-view correspondences. Affine transformations are computed from each pair of corresponding regions to define pixel matches. Pixels matches over consecutive image pairs are concatenated to obtain pixel-level motion trajectories across the image sequence. Motion patterns are learned from the extracted trajectories using a time-delay neural network. We apply the proposed method to recognize 40 hand gestures of American Sign Language. Experimental results show that motion patterns of hand gestures can be extracted and recognized accurately using motion trajectories. Index Terms--Motion segmentation, motion analysis, motion trajectory, American Sign Language, hand gesture recognition, time-delay neural network.

Byline: John T.E. Richardson (1), Alan Woodley (2) Keywords: approaches to studying; Approaches to Studying Inventory; deaf students; distance education; hearing loss; sign language Abstract: This investigation examined approaches to studyingamong deaf students taking courses by distance learning whopreferred to communicate using either sign language or spokenlanguage. In comparison with hearing students, the deaf studentsobtained higher scores on comprehension learning, surfaceapproach, improvidence and fear of failure. Whilst they obtainedhigher scores on reproducing orientation, their qualitativeresponses indicated that this was not because they had beendriven to use rote memorisation. In addition, the deaf studentsseemed just as capable as the hearing students of adopting ameaning orientation. In the specific context of distanceeducation, there were no differences in approaches to studyingrelated to the students' preferred mode of communication.However, communicating by sign language rather than speech haddifferent practical consequences for the students' effectiveworkload. Author Affiliation: (1) Department of Human Sciences, Brunel University, Uxbridge, Middlesex, UB8 3PH, U.K. (2) Institute of Educational Technology, The Open University, Walton Hall, Milton Keynes, MK7 6AA, U.K. Article History: Registration Date: 10/10/2004

An exploration of topic-comment structure in relation to grammaticization is presented to understand grammatical arrangements in American Sign Language (ASL). Topics include topic-comment structure and principles of the grammaticization of ASL.

We present two real-time hidden Markov model-based systems for recognizing sentence-level continuous American Sign Language (ASL) using a single camera to track the user's unadorned hands. The first system observes the user from a desk mounted camera and achieves 92 percent word accuracy. The second system mounts the camera in a cap worn by the user and achieves 98 percent accuracy (97 percent with an unrestricted grammar). Both experiments use a 40-word lexicon. Index Terms - Gesture recognition, hidden Markov models, wearable computers, sign language, motion and pattern analysis.

The acquisition of the movement aspect of American Sign Language signs was examined longitudinally in 9 young children of deaf parents. During monthly home visits, the parents demonstrated on videotape how their children formed the different signs in their lexicons. The parents also demonstrated how they formed or modeled these same signs. Overall, the children correctly produced 61.4% of the movements that were present in the adult sign models. Although the production accuracy of the movement aspect of signs did not improve over the course of the study, the number and complexity of movements produced by the children did increase as they got older and their vocabularies grew in size. Of the different sign movements, contacting action was by far the most frequently produced. The children were also relatively successful in their production of closing action and downward movement. The order of acquisition for the remaining ASL movements, however, was quite variable, with the exception that bidirectional movements tended to be produced more accurately than unidirectional movements. The relationship between children's early rhythmical motor behaviors and the development of sign movements is discussed. KEY WORDS: American Sign Language, deaf, movement, language acquisition, phonemes

We report a sign length effect in deaf users of American Sign Language that is analogous to the word length effect for speech. Lists containing long signs (signs that traverse relatively long distances) produced poorer memory performance than did lists of short signs (signs that do not change in location). Further, this length effect was eliminated by articulatory suppression (repetitive motion of the hands), and articulatory suppression produced an overall drop in performance. The pattern of results, together with previous findings (Wilson & Emmorey, 1997), provides evidence for a working memory system for sign language that consists of a phonological storage buffer and an articulatory rehearsal mechanism. This indicates a close equivalence of structure between working memory for sign language and working memory for speech. The implications of this equivalence are discussed.

N. Masataka (1996) reported results of an experiment in which 6-month-old deaf infants were presented with videotapes that showed 5 identical scripts signed in Japanese Sign Language by 5 deaf mothers toward their deaf infants or toward their deaf friends. The experiment demonstrated that infants showed greater attentional and affective responsiveness to infant-directed signing than to adult-directed signing. The present study explored the possibility that this same phenomenon might extend to hearing infants who have never been exposed to signed language. When the same stimulus tape as used previously was presented to 45 six-month-old hearing infants, they too showed greater attentional and affective responsiveness to infant-directed sign than to adult-directed sign. This fact suggests that infants are prepared to detect sign motherese characteristics without specific experience in the modality.

This study was designed to evaluate the picture quality requirements for three visual communication modes: speechreading, fingerspelling, and sign language. Video recordings of everyday spoken, fingerspelled, and signed sentences were made, and some recordings were processed using a computer simulation of the IBIDEM technology: a videophone based on a novel type of visual sensor. This retina-like sensor, implemented in the camera, has a high resolution in the central part and a degrading resolution in the peripheral part of the picture. Two independent variables were examined: frame rate (10 and 15 frames per second) and spatial resolution (6000 and 8000 pixels per frame). Twenty-four people who were prelingually deaf participated, 8 in each communication mode. The results showed a marked effect of frame rate on speechreading. Fingerspelling and sign language were not affected by frame rate, and spatial resolution had no effect on any of the three communication modes. KEY WORDS: videotelephony, picture quality, speechreading, fingerspelling, sign language

Byline: Brenda C. Seal (1), John D. Bonvillian (2) Abstract: Sign language production of 14 low-functioning students diagnosed with autistic disorder was examined. Videotapes of the students signing with their teachers were analyzed for frequency and accuracy of sign location, handshape, and movement production. The location aspect of signs was produced more accurately by the subjects than either the handshape or movement aspects. Wide individual differences were evident among the students in the number of signs they produced, accuracy of sign formation, and performance on measures of motor functioning. Students' sign vocabulary size and accuracy of sign formation were highly correlated with their performance on two measures of apraxia and with their fine motor age scores. Author Affiliation: (1) Department of Communication Sciences and Disorders, James Madison University, Harrisonburg, Virginia, 22807 (2) University of Virginia, USA Article History: Registration Date: 08/09/2004

Some researchers do not agree to the universality of the leftward WH-movement in American sign language (ASL) and stress that it should be to the right. Others, however, claim that the use of WH-elements in sentence- or discourse-final positions proves that the WH-movement is a leftward specifier of CP and is not rightward. This is also validated by cross-linguistic generalizations and the use of direct and indirect WH-questions in ASL.

Gesture has been a topic of interest since antiquity because of its relevance to theoretical issues in cognition, language acquisition and conversational processes. Through microanalysis, speech and bodily movements are found to be produced together, thus, becoming two aspects of a single process. Furthermore, recognition of cultural differences in gestures are observed which stem from language differences and influenced by cultural values and historical traditions.

Communication strategies for deaf children should focus on the development of age-appropriate language. A combination of sign and spoken language is the most appropriate communication strategy for deaf children. Families should adopt supportive and positive, rather than directive communication styles, fostering an interactive environment for child's language development. Language development in school-age children, role of language preference, and some other related issues are discussed. Psychological assessment and therapeutic intervention strategies are described.

The study explored the possibility that characteristics in perception of speech sounds found in pre-verbal hearing infants might extend to perception of signed languages in congenitally deaf infants during the same period. Participants were deaf mothers (N = 17) and their deaf infants; 5 of the dyads were used to prepare the stimulus tape later shown to the remaining 12 dyads. To develop the tape, the 5 mothers were observed when reciting identical scripts in Japanese Sign Language either toward their infants or toward their deaf friends. The videotape showed that mothers used signs at a significantly slower tempo when interacting with their infants than when interacting with their friends, often repeating the same sign and exaggerating the movements associated with each sign somewhat. When the stimulus tape was presented, deaf infants showed greater attentional and affective responsiveness to infant-directed signing than to adult-directed signing. The fact suggests that human infants may be equally predisposed to attend to motherese characteristics in speech or sign.

The left hemisphere of the brain dominates acquisition and use of both spoken and sign language. Sign language users with brain lesions on one side show left hemisphere dominance despite the dependence of sign language on visuo-spatial processible rather than temporal information. Both sign and spoken languages are highly structure systems and their neural organization in the hemispheres is identical. This indicates that the left hemisphere dominance is due to a higher-property of the language system rather than its physical characteristics or motor production.

The purpose of this investigation was twofold: (a) to determine if there are changes in specific temporal characteristics of speech that occur during simultaneous communication, and (b) to determine if known temporal rules of spoken English are disrupted during simultaneous communication. Ten speakers uttered sentences consisting of a carrier phrase and experimental CVC words under conditions of: (a) speech, (b) speech combined with signed English, and (c) speech combined with signed English for every word except the CVC word that was fingerspelled. The temporal features investigated included: (a) sentence duration, (b) experimental CVC word duration, (c) vowel duration in experimental CVC words, (d) pause duration before and after experimental CVC words, and (e) consonantal effects on vowel duration. Results indicated that for all durational measures, the speech/sign/fingerspelling condition was longest, followed by the speech/sign condition, with the speech condition being shortest. It was also found that for all three speaking conditions, vowels were longer in duration when preceding voiced consonants than vowels preceding their voiceless cognates, and that a low vowel was longer in duration than a high vowel. These findings indicate that speakers consistently reduced their rate of speech when using simultaneous communication, but did not violate these specific temporal rules of English important for consonant and vowel perception. KEY WORDS: simultaneous communication, signed English, fingerspelling, segment duration, pause duration

One of the natural methods of tactual communication in common use among individuals who are both deaf and blind is the tactual reception of sign language. In this method, the receiver (who is deaf-blind) places a hand (or hands) on the dominant (or both) hand(s) of the signer in order to receive, through the tactual sense, the various formational properties associated with signs. In the study reported here, 10 experienced deaf-blind users of either American Sign Language (ASL) or Pidgin Sign English (PSE) participated in experiments to determine their ability to receive signed materials including isolated signs and sentences. A set of 122 isolated signs was received with an average accuracy of 87% correct. The most frequent type of error made in identifying isolated signs was related to misperception of individual phonological components of signs. For presentation of signed sentences (translations of the English CID sentences into ASL or PSE), the performance of individual subjects ranged from 60-85% correct reception of key signs. Performance on sentences was relatively independent of rate of presentation in signs/sec, which covered a range of roughly 1 to 3 signs/sec. Sentence errors were accounted for primarily by deletions and phonological and semantic/syntactic substitutions. Experimental results are discussed in terms of differences in performance for isolated signs and sentences, differences in error patterns for the ASL and PSE groups, and communication rates relative to visual reception of sign language and other natural methods of tactual communication. KEY WORDS: deaf-blindness, tactual communication, sign language

This study determined whether the long-range outcome of first-language acquisition, when the learning begins after early childhood, is similar to that of second-language acquisition. Subjects were 36 deaf adults who had contrasting histories of spoken and sign language acquisition. Twenty-seven subjects were born deaf and began to acquire American Sign Language (ASL) as a first language at ages ranging from infancy to late childhood. Nine other subjects were born with normal hearing, which they lost in late childhood; they subsequently acquired ASL as a second language (because they had acquired spoken English as a first language in early childhood). ASL sentence processing was measured by recall of long and complex sentences and short-term memory for signed digits. Subjects who acquired ASL as a second language after childhood outperformed those who acquired it as a first language at exactly the same age. In addition, the performance of the subjects who acquired ASL as a first language declined in association with increasing age of acquisition. Effects were most apparent for sentence processing skills related to lexical identification, grammatical acceptability, and memory for sentence meaning. No effects were found for skills related to fine-motor production and pattern segmentation.

Samplings of deaf and hearing sign language interpreters were studied to assess the effects of spoken English and sign language fluency on automaticity in word and sign recognition. The Stroop task was used. Responding in sign took longer and led to more Stroop interference than responding verbally. The inclusion of interpreters showed this response had nothing to do with hearing status. The deaf subjects showed more automaticity in identifying signs than words, with the reverse result for hearing subjects. This result was not affected by language fluency.

The purpose of this study was to investigate the effects of prosodic and gestural cues on children's lexical learning. Acquisition of novel words was examined under linguistic input conditions that varied in terms of rate of speech, stress, and use of supplemental visual cues (i.e., gestures). Sixteen kindergarten children served as subjects in this study, including 8 children with normal language (NL) and 8 children with specific language impairment (SLI). A repeated-measures design was used such that all subjects in both groups participated in each of the three experimental conditions (the Rate, Stress, and Visual Condition). Results indicated that acquisition of novel words by the groups with NL and SLI was significantly affected by alterations in speaking rate and by the use of gestures accompanying spoken language. There were no statistically significant effects for the stress manipulations, although subjects with SLI tended to correctly produce novel words that had received emphatic stress during training more often than words presented with neutral stress. Implications of these findings are discussed with respect to the importance of considering how the manner of presentation of the linguistic signal influences the processing and acquisition of language.

Previous studies investigating the organization of articulatory gestures present conflicting accounts of age-related differences in the execution of the articulatory gestures themselves and in the organization of those gestures. Several methodological differences may help to explain these contradictions: First, different studies have used different measures, all of which reflect vocal-tract activity to varying extents; second, the articulatory gestures being analyzed differed across studies; third, the phonetic composition of syllables has varied; and finally, utterance length, and therefore complexity, has varied across studies. The purpose of this study was to investigate the possibility that the reason these methodological differences have led to contradictory results is because the emergence of mature gestural patterns in children's speech is not uniform. To accomplish this goal, detailed acoustic analyses were performed on schwa-stop-vowel utterances from adults and from children (3, 5, and 7 years of age). Temporal measures showed that some acoustic segments were longer in children's than in adults' samples, whereas others were similar in duration. Formant frequencies indicated that vocal-tract opening and closing achieve adult-like patterns of movement by the age of 3 years, but children's tongue gestures are constrained by phonetic context more than those of adults until at least the age of 7 years. Taken together, these results suggest that the pace of development for learning to produce and to coordinate articulatory gestures is not uniform. Thus, the contradictions in findings among earlier studies may very well reflect differences in choices of measurement and utterances to be analyzed, both of which may lead to evaluations of different aspects of gestural patterning.

A studydetermined the receptive communication abilities of severely and profoundly deaf students using individual or combined communication standards such as audition, lip reading, fingerspelling and sign language. Findings show that severely deaf individuals perform well on all communication conditions except on signing. On the other hand, profoundly deaf students did equally well only on signing conditions. Results were analyzed and their implications on deaf education and communication programs were identified.

Hearing adults produced signed and spoken monologues under conditions of quiet or 80 dB SPL of noise and with their vision unobstructed or obstructed. Their signs were videotaped and a random sample of 24 frames was analyzed in each condition through a computer program that determined the overall distance of the hand from a marker placed on the signer's torso. Vocal intensity was digitized from the tape recordings and analyzed by computer for 1 min of continuous speech in each condition. The visual obstruction had no effect either on the distance of the signs or on the vocal intensity of their speech. The subjects increased vocal intensity by about 55% when the noise was introduced (the usual Lombard effect), but the noise had no effect on the distance of the signs. Sign performance was not influenced by visual feedback, and sign and speech were independent communication systems, even in bilingual speaking and signing subjects.

This study investigated the effects of reduced visibility on the distance of signing hands from the bodies of adult signers who are deaf and on their rates of sign production. Subjects were videotaped as they signed with partners in each of three experimental conditions: unimpeded, moderately impeded, and severely impeded visibility. Visual impedance was created by placing screens between signing partners. The distance of a hand from the body, in terms of lateral, vertical, and forward-depth dimensions, was determined from digitized measurements of single video frames randomly selected from those in which the subject was actively signing. Sign rate was calculated as the number of signsproduced in a given segment of time. Signs produced in the severely impeded visibility condition were found to be made at a slower rate and with a mean hand position further forward and vertically higher than signs produced in the unimpeded visibility condition.

A KEY FEATURE OF HUMAN DEVELOPMENT is the regular onset of vocal babbling well before infants are able to utter recognizable words [1]. Vocal babbling is widely recognized as being [...]

How well can a sequence of frames be represented by a subset of the frames? Video sequences of American Sign language (ASL) were investigated in two modes: dynamic (ordinary video) and static (frames printed side by side on the display). An activity index was used to choose critical frames at event boundaries, times when the difference between successive frames is at a local minimum. Sign intelligibility was measured for 32 experienced ASL signers who viewed individual signs. For full gray-scale dynamic signs activity-index subsampling yielded sequences that were significantly more intelligible than when every mth frame was chosen. This result was even more pronounced for static images. For binary images, the relative advantage of activity subsampling was smaller. We conclude that event bounardies can be defined computationally and that subsampling from even boundaries is better than choosing at regular intervals.

Recent research has examined the early stages of language development in signed and spoken language. In this paper we discuss claims that, in the timing of early language milestones, there is an advantage for the acquisition of signed languages. In particular, we will review the evidence that the emergence of the first signs and of the first two-sign combinations is precocious in signing children as compared to the appearance of the first words and of the first two-word strings in speaking children. We conclude that the evidence for early emergence of the first signs deserves continued attention, but that the evidence for early syntax in signing children is not at all compelling. Finally, we discuss the import of a short-lived sign advantage in early language acquisition, in terms of understanding maturational timing mechanisms more generally. Specifically, we address the question of whether or not a single timing mechanism underlies early milestones in the acquisition of both vocabulary and syntax.

Ted Supalla, an associate professor of linguistics, and Elissa Newport, a professor of psychology at the University of Rochester's Sign Language Research Center, are analyzing sign language to find out universal linguistic principles and analyze the evolution of languages. Their research includes examining the verbs of sign languages and searching for proof that languages can be learned more efficiently during a particular period in childhood and early adolescence. Sign language and spoken language are generated in the same parts of the brain.

Byline: Farid Parvini, Cyrus Shahabi We propose a novel approach for recognising static and dynamic hand gestures by analysing the raw data streams generated by the sensors attached to the human hands. We utilise the concept of 'range of motion' in the movement of fingers and exploit this characteristic to analyse the acquired data for recognising hand signs. Our approach for hand gesture recognition addresses two major problems: user-dependency and device-dependency. Furthermore, we show that our approach neither requires calibration nor involves training. We apply our approach for recognising American Sign Language (ASL) signs and show that more than 75% accuracy in sign recognition can be achieved.

Anthropological researchers have tended to treat deaf people as marked more by pathology than behavior. However, recent studies of deafness have investigated issues of language ideology, community identity, and its formation and maintenance.

Nicaraguan deaf students have created a new sign language, and it has fueled the debate among linguists over how languages are formed When Nicaragua established a school for the deaf [...]

It has long been postulated that language is not purely learned, but arises from an interaction between environmental exposure and innate abilities. The innate component becomes more evident in rare situations in which the environment is markedly impoverished. The present study investigated the language production of a generation of deaf Nicaraguans who had not been exposed to a developed language. We examined the changing use of early linguistic structures (specifically, spatial modulations) in a sign language that has emerged since the Nicaraguan group first came together. In under two decades, sequential cohorts of learners systematized the grammar of this new sign language. We examined whether the systematicity being added to the language stems from children or adults; our results indicate that such changes originate in children aged 10 and younger. Thus, sequential cohorts of interacting young children collectively possess the capacity not only to learn, but also to create, language.

Research suggests that people with intellectual impairments who use manual signs to augment or substitute for speech rarely progress beyond the stage of single signs and that word order is particularly problematic. However, the majority of studies have focused on experimental tasks, and relatively little is known about spontaneous sign production in naturalistic settings. The present study explored the linguistic development in sign and speech of 10 children who relied on manual signs (the Makaton vocabulary) as their main means of communication. Mean utterance length in sign ranged from 1.0 to 2.5, and analysis of semantic relations, lexical development, and word order suggested that the children had not developed their language beyond MLU Stage I. Examination of their abilities within the modality of sign indicated that some children were able to manipulate features of sign at a sublexical level. The results are discussed in relation to the language input by teachers, and inferences are drawn regarding the underlying modality of linguistic representation in children who use manual signs. KEY WORDS: intellectual impairment, language development, manual sign, augmentative and alternative communication, word order., Studies of atypical populations provide us with naturally occurring situations to test hypotheses about the generality of development of linguistic skills (Hodapp & Burack, 1990). Children with intellectual impairments who [...]

Evidence from studies of the processing of topographic and classifier constructions in sign language sentences provides a model of how a mental scene description can be represented linguistically, but it also raises questions about how this can be related to spatial linguistic descriptions in spoken languages and their processing. This in turn provides in sights into models of the evolution of language.