Search

Conversation requires a substantial amount of coordination between dialogue participants, from managing turn taking to negotiating mutual understanding. Part of this coordination effort surfaces as the reuse of linguistic behaviour across speakers, a process often referred to as alignment. While the presence of linguistic alignment is well documented in the literature, several questions remain open, including the extent to which patterns of reuse across speakers have an impact on the emergence of labelling conventions for novel referents. In this study, we put forward a methodology for automatically detecting shared lemmatised constructions---expressions with a common lexical core used by both speakers within a dialogue---and apply it to a referential communication corpus where participants aim to identify novel objects for which no established labels exist. Our analyses uncover the usage patterns of shared constructions in interaction and reveal that features such as their frequency and the amount of different constructions used for a referent are associated with the degree of object labelling convergence the participants exhibit after social interaction.More generally, the present study shows that automatically detected shared constructions offer a useful level of analysis to investigate the dynamics of reference negotiation in dialogue.

Linguistic strategies humans use for communication are designed to meet the informational needs of their addressees. Speakers not only adapt their speech but also increase the rate of iconic gestures to enhance the clarity of a message for children. Although sign languages allow signers to take advantage of iconicity far more than what is possible in speech, little is known about whether signers use iconicity as a strategy in child-directed communication. In the present study, we used automatic video pose estimation to analyze descriptions aimed at a child (12yo) vs. an adult produced by 7 deaf adult signers of Italian Sign Language. Overall, signers used iconic constructions more than lexical signs and with comparable frequency in descriptions for both age groups. However, iconic constructions were longer in duration for children. Thus, the present study presents the first evidence that, unlike speakers, signers do not modulate quantitative but only qualitative aspects of iconicity for children.

Children frequently use under-informative expressions (e.g., Side) while describing Left-Right relations between objects but use gestures to disambiguate the relative locations of objects (Karadöller et al., 2022). Here we ask how children collect visual information about the spatial relations they express when planning such descriptions. Twenty Turkish-speaking 8-year-olds saw displays with four pictures of the same two objects in various spatial configurations. Target pictures described to a confederate depicted left-right relations (e.g., lemon left to box). Descriptions were coded whether they were informative in speech, informative with gesture, or under-informative. Children had more target fixations when planning (1) informative than under-informative descriptions (β=0.515, SE=0.131, p

Decades of research have revealed that spatial language is the result of a complex interplay between language-independent, conceptual factors and language-specific forces (e.g., Bowerman, 1996; Johnston & Slobin, 1979; Landau & Jackendoff, 1993; Levinson & Meira 2003; Levinson & Wilkins, 2006). However, currently, a growing body of research acknowledges the importance of a further, much less explored but highly important factor having to do with pragmatic pressures – the general communicative need to convey informative meanings with appropriate levels of required effort (Grice, 1975; Zipf, 1949). This symposium aims to present research documenting the effect of communicative/pragmatic pressures on how spatial language systems are organized, used by speakers of different languages and acquired during development. The contributed papers explore a variety of spatial language phenomena, across many different languages, using a variety of empirical methods and diverse populations of participants.

There is a strong relation between children’s exposure to spatial terms and their later memory accuracy. In the current study, we tested whether the production of spatial terms by children themselves predicts memory accuracy and whether and how language modality of these encodings modulates memory accuracy differently. Hearing child speakers of Turkish and deaf child signers of Turkish Sign Language described pictures of objects in various spatial relations to each other and later tested for their memory accuracy of these pictures in a surprise memory task. We found that having described the spatial relation between the objects predicted better memory accuracy. However, the modality of these descriptions in sign, speech, or speech-plus-gesture did not reveal differences in memory accuracy. We discuss the implications of these findings for the relation between spatial language, memory, and the modality of encoding.

Observing gestures facilitate listeners’ comprehension, especially for visual-spatial information. However, people differ in how and to what extent they benefit from gestures depending on their visual-spatial abilities. This study examined whether and how spatial skills (i.e., mental rotation) relate to how much listeners benefit from observing gestures. We tested 51 Turkish-speaking adults’ comprehension of spatial relations when the critical spatial information was provided in three different conditions: (1) only in speech (e.g., saying “right” without making any gesture), (2) both in speech and in gesture (saying “right” and gesturing to the right), and (3) only in gesture (saying “here” and gesturing to the right). We found that mental rotation scores were associated with increased accuracy only for speech + gesture (z = 2.37, p = .02) and the gesture-only conditions (z = 2.13, p = .03). These results suggest that visual-spatial cognitive resources might be important for gesture processing.

Linguistic expressions of locative spatial relations in sign languages are mostly visually motivated representations of space involving mapping of entities and spatial relations between them onto the hands and the signing space. These are also morphologically complex forms. It is debated whether modality-specific aspects of spatial expressions modulate spatial language development differently in signing compared to speaking children. In a picture description task, we compared the use of locative expressions for containment, support, and occlusion relations by deaf children acquiring Turkish Sign Language and hearing children acquiring Turkish (age 3;5-9;11). Unlike previous reports suggesting a boosting effect of iconicity, and/or a hindering effect of morphological complexity of the locative forms in sign languages, our results show similar developmental patterns for signing and speaking children's acquisition of these forms. Our results suggest the primacy of cognitive development guiding the acquisition of locative expressions by speaking and signing children.

Purpose: This study investigated whether and to what extent iconic co-speech gestures contribute to information from visible speech to enhance degraded speech comprehension at different levels of noise-vocoding. Previous studies of the contributions of these 2 visual articulators to speech comprehension have only been performed separately. Method: Twenty participants watched videos of an actress uttering an action verb and completed a free-recall task. The videos were presented in 3 speech conditions (2- band noise-vocoding, 6-band noise-vocoding, clear), 3 multimodal conditions (speech + lips blurred, speech + visible speech, speech + visible speech + gesture), and 2 visual-only conditions (visible speech, visible speech + gesture). Results: Accuracy levels were higher when both visual articulators were present compared with 1 or none. The enhancement effects of (a) visible speech, (b) gestural information on top of visible speech, and (c) both visible speech and iconic gestures were larger in 6-band than 2-band noise-vocoding or visual-only conditions. Gestural enhancement in 2-band noise-vocoding did not differ from gestural enhancement in visual-only conditions. Conclusions: When perceiving degraded speech in a visual context, listeners benefit more from having both visual articulators present compared with 1. This benefit was larger at 6-band than 2-band noise-vocoding, where listeners can benefit from both phonological cues from visible speech and semantic cues from iconic gestures to disambiguate speech.

Languages typically express semantic components of motion events such as manner (roll) and path (down) in separate lexical items. We explore how these combinatorial possibilities of language arise by focusing on (i) gestures produced by deaf children who lack access to input from a conventional language (homesign); (ii) gestures produced by hearing adults and children while speaking; and (iii) gestures used by hearing adults without speech when asked to do so in elicited descriptions of motion events with simultaneous manner and path. Homesigners tended to conflate manner and path in one gesture, but also used a mixed form, adding a manner and/or path gesture to the conflated form sequentially. Hearing speakers, with or without speech, used the conflated form, gestured manner, or path, but rarely used the mixed form. Mixed form may serve as an intermediate structure on the way to the discrete and sequenced forms found in natural languages.

Does spatial language influence how people think about space? To address this question, we observed children who did not know a conventional language, and tested their performance on nonlinguistic spatial tasks. We studied deaf children living in Istanbul whose hearing losses prevented them from acquiring speech and whose hearing parents had not exposed them to sign. Lacking a conventional language, the children used gestures, called "homesigns", to communicate. In Study 1, we asked whether homesigners used gesture to convey spatial relations, and found that they did not. In Study 2, we tested a new group of homesigners on a Spatial Mapping Task, and found that they performed significantly worse than hearing Turkish children who were matched to the deaf children on another cognitive task. The absence of spatial language thus went hand-in-hand with poor performance on the nonlinguistic spatial task, pointing to the importance of spatial language in thinking about space. (Contains 5 figures.)

Speakers choose a particular expression based on many factors, including availability of the referent in the perceptual context. We examined whether, when expressing referents, monolingual English- and Turkish-speaking children: (1) are sensitive to perceptual context, (2) express this sensitivity in language-specific ways, and (3) use co-speech gestures to specify referents that are underspecified. We also explored the mechanisms underlying children's sensitivity to perceptual context. Children described short vignettes to an experimenter under two conditions: The characters in the vignettes were present in the perceptual context ("perceptual context"); the characters were absent ("no perceptual context"). Children routinely used nouns in the "no perceptual context" condition, but shifted to pronouns (English-speaking children) or omitted arguments (Turkish-speaking children) in the "perceptual context" condition. Turkish-speaking children used underspecified referents more frequently than English-speaking children in the "perceptual context" condition; however, they compensated for the difference by using gesture to specify the forms. Gesture thus gives children learning structurally different languages a way to achieve comparable levels of specification while at the same time adhering to the referential expressions dictated by their language. (Contains 3 figures and 14 footnotes.)

The way adults express manner and path components of a motion event varies across typologically different languages both in speech and cospeech gestures, showing that language specificity in event encoding influences gesture. The authors tracked when and how this multimodal cross-linguistic variation develops in children learning Turkish and English, 2 typologically distinct languages. They found that children learn to speak in language-specific ways from age 3 onward (i.e., English speakers used 1 clause and Turkish speakers used 2 clauses to express manner and path). In contrast, English- and Turkish-speaking children's gestures looked similar at ages 3 and 5 (i.e., separate gestures for manner and path), differing from each other only at age 9 and in adulthood (i.e., English speakers used 1 gesture, but Turkish speakers used separate gestures for manner and path). The authors argue that this pattern of the development of cospeech gestures reflects a gradual shift to language-specific representations during speaking and shows that looking at speech alone may not be sufficient to understand the full process of language acquisition. (Contains 5 figures, 2 tables, and 2 footnotes.)

Pragmatic development requires the ability to use linguistic forms, along with non-verbal cues, to focus an interlocutor's attention on a referent during conversation. We investigate the development of this ability by examining how the use of demonstratives is learned in Turkish, where a three-way demonstrative system ("bu," "su," o) obligatorily encodes both distance contrasts (i.e. proximal and distal) and absence or presence of the addressee's visual attention on the referent. A comparison of the demonstrative use by Turkish children (6 four- and 6 six-year-olds) and 6 adults during conversation shows that adultlike use of attention directing demonstrative, "su," is not mastered even at the age of six, while the distance contrasts are learned earlier. This language specific development reveals that designing referential forms in consideration of recipient's attentional status during conversation is a pragmatic feat that takes more than six years to develop. (Contains 1 footnote.) [This research was supported by the Turkish Academy of Sciences.]

Examines how undergraduate readers monitor and evaluate the concerns of characters over the course of a narrative. Discusses what kinds of evaluation the reader makes, what the reader evaluates, the functions that these evaluative inferences serve in comprehension, and the multiple perspectives (character, narrator, or presenter) taken by the reader during reading and comprehending of narratives. (SR)

Investigates how Turkish children of different ages discuss a conversation they have witnessed during a televised dialog between two Sesame Street characters. Findings reveal that there is a developmental change in children's ability to use appropriate linguistic means to orient their listeners to the different properties of a conversation. (17 references) (Author/CK)

To test whether the language we speak influences our behavior even when we are not speaking, we asked speakers of four languages differing in their predominant word orders (English, Turkish, Spanish, and Chinese) to perform two nonverbal tasks: a communicative task (describing an event by using gesture without speech) and a noncommunicative task (reconstructing an event with pictures). We found that the word orders speakers used in their everyday speech did not influence their nonverbal behavior. Surprisingly, speakers of all four languages used the same order and on both nonverbal tasks. This order, actor--patient--act, is analogous to the subject--object--verb pattern found in many languages of the world and, importantly, in newly developing gestural languages. The findings provide evidence for a natural order that we impose on events when describing and reconstructing them nonverbally and exploit when constructing language anew. gesture | language genesis | sign language | word order

A new sign language has been created by deaf Nicaraguans over the past 25 years, providing an opportunity to observe the inception of universal hallmarks of language. We found that in their initial creation of the language, children analyzed complex events into basic elements and sequenced these elements into hierarchically structured expressions according to principles not observed in gestures accompanying speech in the surrounding language. Successive cohorts of learners extended this procedure, transforming Nicaraguan signing from its early gestural form into a linguistic system. We propose that this early segmentation and recombination reflect mechanisms with which children learn, and thereby perpetuate, language. Thus, children naturally possess learning abilities capable of giving language its fundamental structure., Certain properties of language are so central to the way languages operate, and so widely observed, that Hockett termed them 'design features' of language (1). Tins study asks whether these [...]

The cognitive process that leads to gesture production and the relationship between the latter and speech production is investigated through a cross-linguistic comparison in Turkish, Japanese, and English. The findings indicate that spatio-motoric processes that interact with the speech production process generate gestures.

The question of whether gestures during speech go beyond being an aid to the speaker to being a means of communication with the addressee, is examined. It is concluded that speakers do use gestures to communicate, which supports the view that gestures are a part of language use.

During face-to-face communication, one does not only hear speech but also see a speaker's communicative hand movements. It has been shown that such hand gestures play an important role in communication where the two modalities influence each other's interpretation. A gesture typically temporally overlaps with coexpressive speech, but the gesture is often initiated before (but not after) the coexpressive speech. The present ERP study investigated what degree of asynchrony in the speech and gesture onsets are optimal for semantic integration of the concurrent gesture and speech. Videos of a person gesturing were combined with speech segments that were either semantically congruent or incongruent with the gesture. Although gesture and speech always overlapped in time, gesture and speech were presented with three different degrees of asynchrony. In the SOA 0 condition, the gesture onset and the speech onset were simultaneous. In the SOA 160 and 360 conditions, speech was delayed by 160 and 360 msec, respectively. ERPs time locked to speech onset showed a significant difference between semantically congruent versus incongruent gesture-speech combinations on the N400 for the SOA 0 and 160 conditions. No significant difference was found for the SOA 360 condition. These results imply that speech and gesture are integrated most efficiently when the differences in onsets do not exceed a certain time span because of the fact that iconic gestures need speech to be disambiguated in a way relevant to the speech context.