Search

Affective behaviors enable social robots to not only establish better connections with humans but also serve as a tool for the robots to express their internal states. It has been well established that emotions are important to signal understanding in Human-Robot Interaction (HRI). This work aims to harness the power of Large Language Models (LLM) and proposes an approach to control the affective behavior of robots. By interpreting emotion appraisal as an Emotion Recognition in Conversation (ERC) tasks, we used GPT-3.5 to predict the emotion of a robot’s turn in real-time, using the dialogue history of the ongoing conversation. The robot signaled the predicted emotion using facial expressions. The model was evaluated in a within-subjects user study (N = 47) where the model-driven emotion generation was compared against conditions where the robot did not display any emotions and where it displayed incongruent emotions. The participants interacted with the robot by playing a card sorting game that was specifically designed to evoke emotions. The results indicated that the emotions were reliably generated by the LLM and the participants were able to perceive the robot’s emotions. It was found that the robot expressing congruent model-driven facial emotion expressions were perceived to be significantly more human-like, emotionally appropriate, and elicit a more positive impression. Participants also scored significantly better in the card sorting game when the robot displayed congruent facial expressions. From a technical perspective, the study shows that LLMs can be used to control the affective behavior of robots reliably in real-time. Additionally, our results could be used in devising novel human-robot interactions, making robots more effective in roles where emotional interaction is important, such as therapy, companionship, or customer service.

Abstract It is currently difficult to successfully choose the correct type of antidepressant for individual patients. To discover patterns in patient characteristics, treatment choices and outcomes, we performed retrospective Bayesian network analysis combined with natural language processing (NLP). This study was conducted at two mental healthcare facilities in the Netherlands. Adult patients admitted and treated with antidepressants between 2014 and 2020 were included. Outcome measures were antidepressant continuation, prescription duration and four treatment outcome topics: core complaints, social functioning, general well-being and patient experience, extracted through NLP of clinical notes. Combined with patient and treatment characteristics, Bayesian networks were constructed at both facilities and compared. Antidepressant choices were continued in 66% and 89% of antidepressant trajectories. Score-based network analysis revealed 28 dependencies between treatment choices, patient characteristics and outcomes. Treatment outcomes and prescription duration were tightly intertwined and interacted with antipsychotics and benzodiazepine co-medication. Tricyclic antidepressant prescription and depressive disorder were important predictors for antidepressant continuation. We show a feasible way of pattern discovery in psychiatry data, through combining network analysis with NLP. Further research should explore the found patterns in patient characteristics, treatment choices and outcomes prospectively, and the possibility of translating these into a tool for clinical decision support.

Is language unique in how it evokes conceptual content, and if so, why? In an influential study, Lupyan & Thompson-Schill (2012) report that labels (nouns like “dog”) have a stronger cuing effect in picture verification tasks than “equally informativeand predictive nonverbal cues”, like the sound of a barking dog. Here we sought to better understand the factors that lead to a label advantage. First, while we replicate the label advantage itself, our data do not support the assumption that labels and environmentalsounds are equally informative. Instead, we show that different cue types are associated with target images to different degrees, and that labels show the strongest association. Moreover, the degree of association is a better predictor of reaction times than cue type. Thus, we conclude that labels are not more effective at activating the same semantic content than non-verbal cues, but rather activate different semantic content.

Abstract Background Developing predictive models for precision psychiatry is challenging because of unavailability of the necessary data: extracting useful information from existing electronic health record (EHR) data is not straightforward, and available clinical trial datasets are often not representative for heterogeneous patient groups. The aim of this study was constructing a natural language processing (NLP) pipeline that extracts variables for building predictive models from EHRs. We specifically tailor the pipeline for extracting information on outcomes of psychiatry treatment trajectories, applicable throughout the entire spectrum of mental health disorders (“transdiagnostic”). Methods A qualitative study into beliefs of clinical staff on measuring treatment outcomes was conducted to construct a candidate list of variables to extract from the EHR. To investigate if the proposed variables are suitable for measuring treatment effects, resulting themes were compared to transdiagnostic outcome measures currently used in psychiatry research and compared to the HDRS (as a gold standard) through systematic review, resulting in an ideal set of variables. To extract these from EHR data, a semi-rule based NLP pipeline was constructed and tailored to the candidate variables using Prodigy. Classification accuracy and F1-scores were calculated and pipeline output was compared to HDRS scores using clinical notes from patients admitted in 2019 and 2020. Results Analysis of 34 questionnaires answered by clinical staff resulted in four themes defining treatment outcomes: symptom reduction, general well-being, social functioning and personalization. Systematic review revealed 242 different transdiagnostic outcome measures, with the 36-item Short-Form Survey for quality of life (SF36) being used most consistently, showing substantial overlap with the themes from the qualitative study. Comparing SF36 to HDRS scores in 26 studies revealed moderate to good correlations (0.62—0.79) and good positive predictive values (0.75—0.88). The NLP pipeline developed with notes from 22,170 patients reached an accuracy of 95 to 99 percent (F1 scores: 0.38 – 0.86) on detecting these themes, evaluated on data from 361 patients. Conclusions The NLP pipeline developed in this study extracts outcome measures from the EHR that cater specifically to the needs of clinical staff and align with outcome measures used to detect treatment effects in clinical trials.

During listening, brain activity tracks the rhythmic structures of speech signals. Here, we directly dissociated the contribution of neural envelope tracking in the processing of speech acoustic cues from that related to linguistic processing. We examined the neural changes associated with the comprehension of Noise-Vocoded (NV) speech using magnetoencephalography (MEG). Participants listened to NV sentences in a 3-phase training paradigm: (1) pre-training, where NV stimuli were barely comprehended, (2) training with exposure of the original clear version of speech stimulus, and (3) post-training, where the same stimuli gained intelligibility from the training phase. Using this paradigm, we tested if the neural responses of a speech signal was modulated by its intelligibility without any change in its acoustic structure. To test the influence of spectral degradation on neural envelope tracking independently of training, participants listened to two types of NV sentences (4-band and 2-band NV speech), but were only trained to understand 4-band NV speech. Significant changes in neural tracking were observed in the delta range in relation to the acoustic degradation of speech. However, we failed to find a direct effect of intelligibility on the neural tracking of speech envelope in both theta and delta ranges, in both auditory regions-of-interest and whole-brain sensor-space analyses. This suggests that acoustics greatly influence the neural tracking response to speech envelope, and that caution needs to be taken when choosing the control signals for speech-brain tracking analyses, considering that a slight change in acoustic parameters can have strong effects on the neural tracking response.

Abstract Background Age-related changes in muscle properties affect daily functioning, therefore a reliable assessment of such properties is required. We examined the effects of age on reliability, muscle quality and interrelation among muscle architecture (MA) parameters of the gastrocnemius medialis (GM), tibialis anterior (TA), and vastus lateralis (VL) muscles. Methods Three raters scored ultrasound (US) scans of 12 healthy younger and older adults, on fascicle length (FL), pennation angle (PA) and muscle thickness (MT). Intra- and inter-rater reliability of MA measures in rest and contraction was assessed by intraclass correlation coefficients (ICC) and standard error of measurements (SEM, SEM%). The relationship between MA parameters was examined using Pearson correlation coefficients. Muscle quality (MQ) was examined using mean pixel intensity. Results Reliability was moderate to excellent for TA in both groups (ICCs: 0.64–0.99, SEM% = 1.6–14.8%), and for VL in the younger group (ICCs: 0.67–0.98, SEM% = 2.0–18.3%). VL reliability was poor to excellent in older adults (ICCs: 0.22–0.99, SEM% = 2.7–36.0%). For GM, ICCs were good to excellent (ICCs: 0.76–0.99) in both groups, but GM SEM% were higher in older adults (SEM%Younger = 1.5–10.7%, SEM%Older = 1.6–28.1%). Muscle quality was on average 19.0% lower in older vs. younger adults. In both groups, moderate to strong correlations were found for VL FL and MT (r ≥ 0.54), and TA PA and MT (r ≥ 0.72), while TA FL correlated with MT (r ≥ 0.67) in younger adults only. Conclusions In conclusion, age- and muscle-specificities were present in the relationships between MT and PA, and MT and FL at rest. Furthermore, the reliability of MA parameters assessed with 2D panoramic US is acceptable. However, the level of reliability varies with age, muscle and MA measure. In older adults notably, the lowest reliability was observed in the VL muscle. Among the MA parameters, MT appears to be the simplest and most easily reproducible parameter in all muscles and age groups.

AbstractThis study investigated two questions. One is: To what degree is sentence processing beyond single words independent of the input modality (speech vs. reading)? The second question is: Which parts of the network recruited by both modalities is sensitive to syntactic complexity? These questions were investigated by having more than 200 participants read or listen to well-formed sentences or series of unconnected words. A largely left-hemisphere frontotemporoparietal network was found to be supramodal in nature, i.e., independent of input modality. In addition, the left inferior frontal gyrus (LIFG) and the left posterior middle temporal gyrus (LpMTG) were most clearly associated with left-branching complexity. The left anterior temporal lobe showed the greatest sensitivity to sentences that differed in right-branching complexity. Moreover, activity in LIFG and LpMTG increased from sentence onset to end, in parallel with an increase of the left-branching complexity. While LIFG, bilateral anterior temporal lobe, posterior MTG, and left inferior parietal lobe all contribute to the supramodal unification processes, the results suggest that these regions differ in their respective contributions to syntactic complexity related processing. The consequences of these findings for neurobiological models of language processing are discussed.

IntroductionMood is a constant in our daily life and can permeate all levels of cognition. We examined whether and how mood influences the processing of discourse content that is relatively neutral and not loaded with emotion. During discourse processing, readers have to constantly strike a balance between what they know in long term memory and what the current discourse is about. Our general hypothesis is that mood states would affect this balance. We hypothesized that readers in a positive mood would rely more on default world knowledge, whereas readers in a negative mood would be more inclined to analyze the details in the current discourse.MethodsParticipants were put in a positive and a negative mood via film clips, one week apart. In each session, after mood manipulation, they were presented with sentences in discourse materials. We created sentences such as “With the lights on you can see...” that end with critical words (CWs) “more” or “less”, where general knowledge supports “more”, not “less”. We then embedded each of these sentences in a wider discourse that does/does not support the CWs (a story about driving in the night vs. stargazing). EEG was recorded throughout.ResultsThe results showed that first, mood manipulation was successful in that there was a significant mood difference between sessions. Second, mood did not modulate the N400 effects. Participants in both moods detected outright semantic violations and allowed world knowledge to be overridden by discourse context. Third, mood modulated the LPC (Late Positive Component) effects, distributed in the frontal region. In negative moods, the LPC was sensitive to one-level violation. That is, CWs that were supported by only world knowledge, only discourse, and neither, elicited larger frontal LPCs, in comparison to the condition where CWs were supported by both world knowledge and discourse.DiscussionThese results suggest that mood does not influence all processes involved in discourse processing. Specifically, mood does not influence lexical-semantic retrieval (N400), but it does influence elaborative processes for sensemaking (P600) during discourse processing. These results advance our understanding of the impact and time course of mood on discourse.

Abstract Diffusible iodine-based contrast-enhanced computed tomography (diceCT) is progressively used in clinical and morphological research to study developmental anatomy. Lugol’s solution (Lugol) has gained interest as an effective contrast agent; however, usage is limited due to extensive soft-tissue shrinkage. The mechanism of Lugol-induced shrinkage and how to prevent it is largely unknown, hampering applications of Lugol in clinical or forensic cases where tissue shrinkage can lead to erroneous diagnostic conclusions. Shrinkage was suggested to be due to an osmotic imbalance between tissue and solution. Pilot experiments pointed to acidification of Lugol, but the relation of acidification and tissue shrinkage was not evaluated. In this study, we analyzed the relation between tissue shrinkage, osmolarity and acidification of the solution during staining. Changes in tissue volume were measured on 2D-segmented magnetic resonance and diceCT images using AMIRA software. Partial correlation and stepwise regression analysis showed that acidification of Lugol is the main cause of tissue shrinkage. To prevent acidification, we developed a buffered Lugol’s solution (B-Lugol) and showed that stabilizing its pH almost completely prevented shrinkage without affecting staining. Changing from Lugol to B-Lugol is a major improvement for clinical and morphological research and only requires a minor adaptation of the staining protocol.

Abstract Linguistic labels exert a particularly strong top-down influence on perception. The potency of this influence has been ascribed to their ability to evoke category-diagnostic features of concepts. In doing this, they facilitate the formation of a perceptual template concordant with those features, effectively biasing perceptual activation towards the labelled category. In this study, we employ a cueing paradigm with moving, point-light stimuli across three experiments, in order to examine how the number of biological motion features (form and kinematics) encoded in lexical cues modulates the efficacy of lexical top-down influence on perception. We find that the magnitude of lexical influence on biological motion perception rises as a function of the number of biological motion-relevant features carried by both cue and target. When lexical cues encode multiple biological motion features, this influence is robust enough to mislead participants into reporting erroneous percepts, even when a masking level yielding high performance is used.

New linguistic information must be integrated into our existing language system. Using a novel experimental task that incorporates a syntactic priming paradigm into artificial language learning, we investigated how new grammatical regularities and words are learned. This innovation allowed us to control the language input the learner received, while the syntactic priming paradigm provided insight into the nature of the underlying syntactic processing machinery. The results of the present study pointed to facilitatory syntactic processing effects within the first days of learning: Syntactic and lexical priming effects revealed participants' sensitivity to both novel words and word orders. This suggested that novel syntactic structures and their meaning (form-function mapping) can be acquired rapidly through incidental learning. More generally, our study indicated similar mechanisms for learning and processing in both artificial and natural languages, with implications for the relationship between first and second language learning.

Studies of sound symbolism have shown that people canassociate sound and meaning in consistent ways whenpresented with maximally contrastive stimulus pairs ofnonwords such as bouba/kiki (rounded/sharp) or mil/mal(small/big). Recent work has shown the effect extends toantonymic words from natural languages and has proposed arole for shared cross-modal correspondences in biasing form-to-meaning associations. An important open question is howthe associations work, and particularly what the role is ofsound-symbolic matches versus mismatches. We report on alearning task designed to distinguish between three existingtheories by using a spectrum of sound-symbolically matching,mismatching, and neutral (neither matching nor mismatching)stimuli. Synthesized stimuli allow us to control for prosody,and the inclusion of a neutral condition allows a direct test ofcompeting accounts. We find evidence for a sound-symbolicmatch boost, but not for a mismatch difficulty compared tothe neutral condition.

Letters are more easily identified when embedded in a word. Here, the authors show that word contexts can enhance letter information in early visual cortex, suggesting that the advantage offered by words occurs already during early perceptual processing.

Listening to speech is difficult in noisy environments, and is even harder when the interfering noise consists of intelligible speech as compared to unintelligible sounds. This suggests that the competing linguistic information interferes with the neural processing of target speech. Interference could either arise from a degradation of the neural representation of the target speech, or from increased representation of distracting speech that enters in competition with the target speech. We tested these alternative hypotheses using magnetoencephalography (MEG) while participants listened to a target clear speech in the presence of distracting noise-vocoded speech. Crucially, the distractors were initially unintelligible but became more intelligible after a short training session. Results showed that the comprehension of the target speech was poorer after training than before training. The neural tracking of target speech in the delta range (1–4 Hz) reduced in strength in the presence of a more intelligible distractor. In contrast, the neural tracking of distracting signals was not significantly modulated by intelligibility. These results suggest that the presence of distracting speech signals degrades the linguistic representation of target speech carried by delta oscillations.

[This corrects the article DOI: 10.1371/journal.pone.0260952.].

Abstract Background Pelvic organ prolapse (POP) affects up to 40% of parous women which adversely affects the quality of life. During a life time, 20% of all women will undergo an operation. In general the guidelines advise a vaginal operation in case of uterine descent: hysterectomy with uterosacral ligament plication (VH), sacrospinous hysteropexy (SSH) or a modified Manchester operation (MM). In the last decade, renewed interest in uterus sparing techniques has been observed. Previous studies have shown non-inferiority between SSH and VH. Whether or not SSH and MM are comparable concerning anatomical and functional outcome is still unknown. The practical application of both operations is at least in The Netherlands a known cause of practice pattern variation (PPV). To reveal any difference between both techniques the SAM-study was designed. Methods The SAM-study is a randomized controlled multicentre non-inferiority study which compares SSH and MM. Women with symptomatic POP in any stage, uterine descent and POP-Quantification (POP-Q) point D at ≤ minus 1 cm are eligible. The primary outcome is the composite outcome at two years of absence of prolapse beyond the hymen in any compartment, the absence of bulge symptoms and absence of reoperation for pelvic organ prolapse. Secondary outcomes are hospital parameters, surgery related morbidity/complications, pain perception, further treatments for prolapse or urinary incontinence, POP-Q anatomy in all compartments, quality-of-life, sexual function, and cost-effectiveness. Follow-up takes place at 6 weeks, 12 and 24 months. Additionally at 12 weeks, 6 and 9 months cost-effectiveness will be assessed. Validated questionnaires will be used and gynaecological examination will be performed. Analysis will be performed following the intention-to-treat and per protocol principle. With a non-inferiority margin of 9% and an expected loss to follow-up of 10%, 424 women will be needed to prove non-inferiority with a confidence interval of 95%. Discussion This study will evaluate the effectiveness and costs of SSH versus MM in women with primary POP. The evidence will show whether the existing PPV is detrimental and a de-implementation process regarding one of the operations is needed. Trial registration Dutch Trial Register (NTR 6978, http://www.trialregister.nl). Date of registration: 29 January 2018. Prospectively registered.

The size and growth patterns of the components of the human embryonic heart have remained largely undefined. To provide these data, three-dimensional heart models were generated from immunohistochemically stained sections of ten human embryonic hearts ranging from Carnegie stage 10 to 23. Fifty-eight key structures were annotated and volumetrically assessed. Sizes of the septal foramina and atrioventricular canal opening were also measured. The heart grows exponentially throughout embryonic development. There was consistently less left than right atrial myocardium, and less right than left ventricular myocardium. We observed a later onset of trabeculation in the left atrium compared to the right. Morphometry showed that the rightward expansion of the atrioventricular canal starts in week 5. The septal foramina are less than 0.1 mm2 and are, therefore, much smaller than postnatal septal defects. This chronological, graphical atlas of the growth patterns of cardiac components in the human embryo provides quantified references for normal heart development. Thereby, this atlas may support early detection of cardiac malformations in the foetus. This article has an associated First Person interview with the first author of the paper.

Using serial block-face scanning electron microscopy, we report on the internal 3D structures of the brown planthopper, Nilaparvata lugens (Hemiptera: Delphacidae) at nanometer resolution for the first time. Within the reconstructed organs and tissues, we found many novel and fascinating internal structures in the planthopper such as naturally occurring three four-way rings connecting adjacent spiracles to facilitate efficient gas exchange, and fungal endosymbionts in a single huge insect cell occupying 22% of the abdomen volume to enable the insect to live on plant sap. To understand the muscle and stylet movement during phloem sap-sucking, the cephalic skeleton and muscles were reconstructed in feeding nymphs. The results revealed an unexpected contraction of the protractors of the stylets and suggested a novel feeding model for the phloem sap-sucking.

The endeavor to understand the human brain has seen more progress in the last few decades than in the previous two millennia. Still, our understanding of how the human brain relates to behavior in the real world and how this link is modulated by biological, social, and environmental factors is limited. To address this, we designed the Healthy Brain Study (HBS), an interdisciplinary, longitudinal, cohort study based on multidimensional, dynamic assessments in both the laboratory and the real world. Here, we describe the rationale and design of the currently ongoing HBS. The HBS is examining a population-based sample of 1,000 healthy participants (age 30-39) who are thoroughly studied across an entire year. Data are collected through cognitive, affective, behavioral, and physiological testing, neuroimaging, bio-sampling, questionnaires, ecological momentary assessment, and real-world assessments using wearable devices. These data will become an accessible resource for the scientific community enabling the next step in understanding the human brain and how it dynamically and individually operates in its bio-social context. An access procedure to the collected data and bio-samples is in place and published on https://www.healthybrainstudy.nl/en/data-and-methods/access. Trail registration: https://www.trialregister.nl/trial/7955.

The suitability of the artificial grammar learning (AGL) paradigm to capture relevant aspects of the acquisition of linguistic structures has been empirically tested in a number of EEG studies. Some have shown a syntax-related P600 component, but it has not been ruled out that the AGL P600 effect is a response to surface features (e.g., subsequence familiarity) rather than the underlying syntax structure. Therefore, in this study, we controlled for the surface characteristics of the test sequences (associative chunk strength) and recorded the EEG before (baseline preference classification) and after (preference and grammaticality classification) exposure to a grammar. After exposure, a typical, centroparietal P600 effect was elicited by grammatical violations and not by unfamiliar subsequences, suggesting that the AGL P600 effect signals a response to structural irregularities. Moreover, preference and grammaticality classification showed a qualitatively similar ERP profile, strengthening the idea that the implicit structural mere-exposure paradigm in combination with preference classification is a suitable alternative to the traditional grammaticality classification test.

This data article presents datasets associated with the research article entitled “The immunological architecture of granulomatous inflammation in central nervous system tuberculosis’’ (Zaharie et al., 2020). The morphology of tuberculosis related granulomas within the central nervous system of human patients was visualized in six different three-dimensional (3D) models. Post-mortem, formalin fixed and paraffin embedded specimens from deceased tuberculous meningitis patients were immunohistochemically stained and 800 serial histologically stained sections were acquired. Images from all sections were obtained with an Olympus BX43 light microscope and structures were identified, labeled and made three-dimensional. The interactive 3D-models allows the user to directly visualize the morphology of the granulomas and to understand the localization of the granulomas. The 3D-models can be used for multiple purposes and provide both an educational source as a gold standard for further animal studies, human research and the development of in silico models on the topic of central nervous system tuberculosis.

Bodily mimicry often makes the mimickee have more positive feelings about the mimicker. Yet, little is known about the causes of mimicry’s social effects. When people mimic each other’s bodily movements face to face, they can either adopt a mirrorwise perspective (moving in the same absolute direction) or an anatomical perspective (moving in the same direction relative to their own bodies). Mirrorwise mimicry maximizes visuo-spatial similarity between the mimicker and mimickee, whereas anatomical mimicry maximizes the similarity in the states of their motor systems. To compare the social consequences of visuo-spatial and motoric similarity, we asked participants to converse with an embodied virtual agent (VIRTUO), who mimicked their head movements either mirrorwise, anatomically, or not at all. Compared to participants who were not mimicked, those who were mimicked mirrorwise tended to rate VIRTUO more positively, but those who were mimicked anatomically rated him more negatively. During face-to-face conversation, mirrorwise and anatomical mimicry have opposite social consequences. Results suggest that visuo-spatial similarity between mimicker and mimickee, not similarity in motor system activity, gives rise to the positive social effects of bodily mimicry.

Major depressive disorder imposes a substantial disease burden worldwide, ranking as the third leading contributor to global disability. In spite of its ubiquity, classifying and treating depression has proven troublesome. One argument put forward to explain this predicament is the heterogeneity of patients diagnosed with the disorder.Recently, many areas of daily life have witnessed the surge of machine learning techniques, computational approaches to elucidate complex patterns in large datasets, which can be employed to make predictions and detect relevant clusters. Due to the multidimensionality at play in the pathogenesis of depression, it is suggested that machine learning could contribute to improving classification and treatment.In this paper, we investigated literature focusing on the use of machine learning models on datasets with clinical variables of patients diagnosed with depression to predict treatment outcomes or find more homogeneous subgroups. Identified studies based on best practices in the field are evaluated. We found 16 studies predicting outcomes (such as remission) and identifying clusters in patients with depression.The identified studies are mostly still in proof-of-concept phase, with small datasets, lack of external validation, and providing single performance metrics. Larger datasets, and models with similar variables present across these datasets, are needed to develop accurate and generalizable models. We hypothesize that harnessing natural language processing to obtain data ‘hidden' in clinical texts might prove useful in improving prediction models. Besides, researchers will need to focus on the conditions to feasibly implement these models to support psychiatrists and patients in their decision-making in practice. Only then we can enter the realm of precision psychiatry.