Search

Contains fulltext : 248311.pdf (Publisher’s version ) (Open Access) When people interact to establish shared symbols for novel objects or concepts, they often rely on multiple communicative modalities as well as on alignment (i.e., cross-participant repetition of communicative behavior). Yet these interactional resources have rarely been studied together, so little is known about if and how people combine multiple modalities in alignment to achieve joint reference. To investigate this, we systematically track the emergence of lexical and gestural alignment in a referential communication task with novel objects. Quantitative analyses reveal that people frequently use a combination of lexical and gestural alignment, and that such multimodal alignment tends to emerge earlier compared to unimodal alignment. Qualitative analyses of the interactional contexts in which alignment emerges reveal how people flexibly deploy lexical and gestural alignment in line with modality affordances and communicative needs. 28 p.

Contains fulltext : 245641.pdf (Publisher’s version ) (Open Access) Recognized as a simple communicative behavior, referential pointing is cognitively complex because it invites a communicator to consider an addressee's knowledge. Although we know referential pointing is affected by addressees' physical location, it remains unclear whether and how communicators' inferences about addressees' mental representation of the interaction space influence sensorimotor control of referential pointing. The communicative perspective-taking task requires a communicator to point at one out of multiple referents either to instruct an addressee which one should be selected (communicative, COM) or to predict which one the addressee will select (non-communicative, NCOM), based on either which referents can be seen (Level-1 perspective-taking, PT1) or how the referents were perceived (Level-2 perspective-taking, PT2) by the addressee. Communicators took longer to initiate the movements in PT2 than PT1 trials, and they held their pointing fingers for longer at the referent in COM than NCOM trials. The novel findings of this study pertain to trajectory control of the pointing movements. Increasing both communicative and perspective-taking demands led to longer pointing trajectories, with an under-additive interaction between those two experimental factors. This finding suggests that participants generate communicative behaviors that are as informative as required rather than overly exaggerated displays, by integrating communicative and perspective-taking information hierarchically during sensorimotor control. This observation has consequences for models of human communication. It implies that the format of communicative and perspective-taking knowledge needs to be commensurate with the movement dynamics controlled by the sensorimotor system. 24 p.

The present study extends neuro-imaging into conversation through studying dialogue comprehension. Conversation entails rapid responses, with negative semiotics for delay. We explored how expectations about the valence of the forthcoming response develop during the silence before the response and whether negative responses have mainly cognitive or social-emotional consequences. EEG-participants listened to questions from a spontaneous spoken corpus, cross-spliced with short/long gaps and “yes”/“no” responses. Preceding contexts biased listeners to expect the eventual response, which was hypothesised to translate to expectations for a shorter or longer gap. “No” responses showed a trend towards an early positivity, suggesting socio-emotional consequences. Within the long gap, expecting a “yes” response led to an earlier negativity, as well as a trend towards stronger theta-oscillations, after 300 milliseconds. This suggests that listeners anticipate/predict “yes” responses to come earlier than “no” responses, showing strong sensitivities to timing, which presumably promote hastening the pace of verbal interaction.

This study investigated the role of contextual and prosodic information in turn-end estimation by means of a button-press task. We presented participants with turns extracted from a corpus of telephone calls visually (i.e., in transcribed form, word-by-word) and auditorily, and asked them to anticipate turn ends by pressing a button. The availability of the previous conversational context was generally helpful for turn-end estimation in short turns only, and more clearly so in the visual task than in the auditory task. To investigate the role of prosody, we examined whether participants in the auditory task pressed the button close to turn-medial points likely to constitute turn ends based on lexico-syntactic information alone. We observed that the vast majority of such button presses occurred in the presence of an intonational boundary rather than in its absence. These results are consistent with the view that prosodic cues in the proximity of turn ends play a relevant role in turn-end estimation.

Conversation is generally characterized by smooth transitions between turns, with only very short gaps. This entails that responders often begin planning their response before the ongoing turn is finished. However, controversy exists about whether they start planning as early as they can, to make sure they respond on time, or as late as possible, to minimize the overlap between comprehension and production planning. Two earlier EEG studies have found neural correlates of response planning (positive ERP and alpha decrease) as soon as listeners could start planning their response, already midway through the current turn. However, in these studies, the questions asked were highly controlled with respect to the position where planning could start (e.g., very early) and required short and easy responses. The present study measured participants’ EEG while an experimenter interviewed them in a spontaneous interaction. Coding the questions in the interviews showed that, under these natural circumstances, listeners can, in principle, start planning a response relatively early, on average after only about one third of the question has passed. Furthermore, ERP results showed a large positivity, interpreted before as an early neural signature of response planning, starting about half a second after the start of the word that allowed listeners to start planning a response. A second neural signature of response planning, an alpha decrease, was not replicated as reliably. In conclusion, listeners appear to start planning their response early during the ongoing turn, also under natural circumstances, presumably in order to keep the gap between turns short and respond on time. These results have several important implications for turn-taking theories, which need to explain how interlocutors deal with the overlap between comprehension and production, how they manage to come in on time, and the sources that lead to variability between conversationalists in the start of planning.

Rapid response latencies in conversation suggest that responders start planning before the ongoing turn is finished. Indeed, an earlier EEG study suggests that listeners start planning their responses to questions as soon as they can (Bögels, S., Magyari, L., & Levinson, S. C. (2015). Neural signatures of response planning occur midway through an incoming question in conversation. Scientific Reports, 5, 12881). The present study aimed to (1) replicate this early planning effect and (2) investigate whether such early response planning incurs a cost on participants’ concurrent comprehension of the ongoing turn. During the experiment participants answered questions from a confederate partner. To address aim (1), the questions were designed such that response planning could start either early or late in the turn. Our results largely replicate Bögels et al. (2015) showing a large positive ERP effect and an oscillatory alpha/beta reduction right after participants could have first started planning their verbal response, again suggesting an early start of response planning. To address aim (2), the confederate's questions also contained either an expected word or an unexpected one to elicit a differential N400 effect, either before or after the start of response planning. We hypothesized an attenuated N400 effect after response planning had started. In contrast, the N400 effects before and after planning did not differ. There was, however, a positive correlation between participants' response time and their N400 effect size after planning had started; quick responders showed a smaller N400 effect, suggesting reduced attention to comprehension and possibly reduced anticipatory processing. We conclude that early response planning can indeed impact comprehension processing.

In conversation, turn transitions between speakers often occur smoothly, usually within a time window of a few hundred milliseconds. It has been argued, on the basis of a button-press experiment [De Ruiter, J. P., Mitterer, H., & Enfield, N. J. (2006). Projecting the end of a speaker's turn: A cognitive cornerstone of conversation. Language, 82(3):515–535], that participants in conversation rely mainly on lexico-syntactic information when timing and producing their turns, and that they do not need to make use of intonational cues to achieve smooth transitions and avoid overlaps. In contrast to this view, but in line with previous observational studies, our results from a dialogue task and a button-press task involving questions and answers indicate that the identification of the end of intonational phrases is necessary for smooth turn-taking. In both tasks, participants never responded to questions (i.e., gave an answer or pressed a button to indicate a turn end) at turn-internal points of syntactic completion in the absence of an intonational phrase boundary. Moreover, in the button-press task, they often pressed the button at the same point of syntactic completion when the final word of an intonational phrase was cross-spliced at that location. Furthermore, truncated stimuli ending in a syntactic completion point but lacking an intonational phrase boundary led to significantly delayed button presses. In light of these results, we argue that earlier claims that intonation is not necessary for correct turn-end projection are misguided, and that research on turn-taking should continue to consider intonation as a source of turn-end cues along with other linguistic and communicative phenomena.

Item does not contain fulltext In contrast to the large amount of dual-task research investigating the coordination of a linguistic and a non-linguistic task, little research has investigated how two linguistic tasks are coordinated. However, such research would greatly contribute to our understanding of how interlocutors combine speech planning and listening in conversation. In three dual-task experiments we studied how participants coordinated the processing of an auditory stimulus (S1), which was either a syllable or a tone, with selecting a name for a picture (S2). Two SOAs, of 0 ms and 1000 ms, were used. To vary the time required for lexical selection and to determine when lexical selection took place, the pictures were presented with categorically related or unrelated distractor words. In Experiment 1 participants responded overtly to both stimuli. In Experiments 2 and 3, S1 was not responded to overtly, but determined how to respond to S2, by naming the picture or reading the distractor aloud. Experiment 1 yielded additive effects of SOA and distractor type on the picture naming latencies. The presence of semantic interference at both SOAs indicated that lexical selection occurred after response selection for S1. With respect to the coordination of S1 and S2 processing, Experiments 2 and 3 yielded inconclusive results. In all experiments, syllables interfered more with picture naming than tones. This is likely because the syllables activated phonological representations also implicated in picture naming. The theoretical and methodological implications of the findings are discussed. 18 p.

This paper reviews the prospects for the cross-fertilization of conversation-analytic (CA) and neurocognitive studies of conversation, focusing on turn-taking. Although conversation is the primary ecological niche for language use, relatively little brain research has focused on interactive language use, partly due to the challenges of using brain-imaging methods that are controlled enough to perform sound experiments, but still reflect the rich and spontaneous nature of conversation. Recently, though, brain researchers have started to investigate conversational phenomena, for example by using 'overhearer' or controlled interaction paradigms. We review neuroimaging studies related to turn-taking and sequence organization, phenomena historically described by CA. These studies for example show early action recognition and immediate planning of responses midway during an incoming turn. The review discusses studies with an eye to a fruitful interchange between CA and neuroimaging research on conversation and an indication of how these disciplines can benefit from each other.

Prosodic breaks (PBs) can indicate a sentence’s syntactic structure. Event-related brain potentials (ERPs) are an excellent way to study auditory sentence processing, since they provide an on-line measure across a complete sentence, in contrast to other on- and off-line methods. ERPs for the first time allowed investigating the processing of a PB itself. PBs reliably elicit a closure positive shift (CPS). We first review several studies on the CPS, leading to the conclusion that it is elicited by abstract structuring or phrasing of the input. Then we review ERP findings concerning the role of PBs in sentence processing as indicated by ERP components like the N400, P600 and LAN. We focus on whether and how PBs can (help to) disambiguate locally ambiguous sentences. Differences in results between different studies can be related to differences in items, initial parsing preferences and tasks. Finally, directions for future research are discussed.

This study addresses the question whether prosodic information can affect the choice for a syntactic analysis in auditory sentence processing. We manipulated the prosody (in the form of a prosodic break; PB) of locally ambiguous Dutch sentences to favor one of two interpretations. The experimental items contained two different types of so-called control verbs (subject and object control) in the matrix clause and were syntactically disambiguated by a transitive or by an intransitive verb. In Experiment 1, we established the default off-line preference of the items for a transitive or an intransitive disambiguating verb with a visual and an auditory fragment completion test. The results suggested that subject- and object-control verbs differently affect the syntactic structure that listeners expect. In Experiment 2, we investigated these two types of verbs separately in an on-line ERP study. Consistent with the literature, the PB elicited a closure positive shift. Furthermore, in subject-control items, an N400 effect for intransitive relative to transitive disambiguating verbs was found, both for sentences with and for sentences without a PB. This result suggests that the default preference for subject-control verbs goes in the same direction as the effect of the PB. In object-control items, an N400 effect for intransitive relative to transitive disambiguating verbs was found for sentences with a PB but no effect in the absence of a PB. This indicates that a PB can affect the syntactic analysis that listeners pursue.

Contains fulltext : 205405.pdf (Publisher’s version ) (Open Access) A striking puzzle about language use in everyday conversation is that turn-taking latencies are usually very short, whereas planning language production takes much longer. This implies overlap between language comprehension and production processes, but the nature and extent of such overlap has never been studied directly. Combining an interactive quiz paradigm with EEG measurements in an innovative way, we show that production planning processes start as soon as possible, that is, within half a second after the answer to a question can be retrieved (up to several seconds before the end of the question). Localization of ERP data shows early activation even of brain areas related to late stages of production planning (e.g., syllabification). Finally, oscillation results suggest an attention switch from comprehension to production around the same time frame. This perspective from interactive language use throws new light on the performance characteristics that language competence involves.

We investigate the timing of pre-answer inbreaths in order to shed light on the time course of response planning and execution in conversational turn-taking. Using acoustic and inductive plethysmography recordings of seven dyadic conversations in Dutch, we show that pre-answer inbreaths in conversation typically begin briefly after the end of questions. We also show that the presence of a pre-answer inbreath usually co-occurs with substantially delayed answers, with a modal latency of 576 vs. 100 ms for answers not preceded by an inbreath. Based on previously reported minimal latencies for internal intercostal activation and the production of speech sounds, we propose that vocal responses, either in the form of a pre-utterance inbreath or of speech proper when an inbreath is not produced, are typically launched in reaction to information present in the last portion of the interlocutor's turn. We also show that short responses are usually made on residual breath, while longer responses are more often preceded by an inbreath. This relation of inbreaths to answer length suggests that by the time an inbreath is launched, typically during the last few hundred milliseconds of the question, the length of the answer is often prepared to some extent. Together, our findings are consistent with a two-stage model of response planning in conversational turn-taking: early planning of content often carried out in overlap with the incoming turn, and late launching of articulation based on the identification of turn-final cues.

Item does not contain fulltext This ERP study investigates whether a superfluous prosodic break (i.e., a prosodic break that does not coincide with a syntactic break) has more severe processing consequences during auditory sentence comprehension than a missing prosodic break (i.e., the absence of a prosodic break at the position of a syntactic break). Participants listened to temporarily ambiguous sentences involving a prosody-syntax match or mismatch. The disambiguation of these sentences was always lexical in nature in the present experiment. This contrasts with a related study by Pauker, Itzhak, Baum, and Steinhauer (2011), where the disambiguation was of a lexical type for missing PBs and of a prosodic type for superfluous PBs. Our results converge with those of Pauker et al. (2011): superfluous prosodic breaks lead to more severe processing problems than missing prosodic breaks. Importantly, the present results extend those of Pauker et al. (2011) showing that this holds when the disambiguation is always lexical in nature. Furthermore, our results show that the way listeners use prosody can change over the course of the experiment which bears consequences for future studies. 14 p.

Contains fulltext : 86301.pdf (author's version ) (Open Access) Previous studies on Cantonese Chinese showed that rising question intonation contours on low-toned words lead to frequent misperceptions of the tones. Here we explored the processing consequences of this interaction between tone and intonation by comparing the processing and identification of monosyllabic critical words at the end of questions and statements, using a tone identification task, and ERPs as an online measure of speech comprehension. Experiment 1 yielded higher error rates for the identification of low tones at the end of questions and a larger N400-P600 pattern, reflecting processing difficulty and reanalysis, compared to other conditions. In Experiment 2, we investigated the effect of immediate lexical context on the tone by intonation interaction. Increasing contextual constraints led to a reduction in errors and the disappearance of the P600-effect. These results indicate that there is an immediate interaction between tone, intonation, and context in online speech comprehension. The difference in performance and activation patterns between the two experiments highlights the significance of context in understanding a tone language, like Cantonese-Chinese. Speech Prosody 2010, 11 mei 2010

independent component analysis and current-source density. Behaviorally, the depressed sample showed less accurate heartbeat perception in comparison to the control group (p=.011). The two groups alsodemonstrated psychophysiological differences, showing that heartbeat-evoked potentials were significantly reduced in depressed patients. Our results suggest that heartbeat evoked potentials are objective markers of altered bodily awareness. Reduced interoception during depression may be linked to alexithymia, as well as to both decreased capacity for decision-making and for cognitive processing. It may be helpful to practice interoceptive awareness to improve depressive symptoms, for example by practicing meditation.