Search

A growing number of social interactions are taking place virtually on videoconferencing platforms. Here, we explore potential effects of virtual interactions on observed behavior, subjective experience, and neural “single-brain” and “interbrain” activity via functional near-infrared spectroscopy neuroimaging. We scanned a total of 36 human dyads (72 participants, 36 males, 36 females) who engaged in three naturalistic tasks (i.e., problem-solving, creative-innovation, socio-emotional task) in either an in-person or virtual (Zoom) condition. We also coded cooperative behavior from audio recordings. We observed reduced conversational turn-taking behavior during the virtual condition. Given that conversational turn-taking was associated with other metrics of positive social interaction (e.g., subjective cooperation and task performance), this measure may be an indicator of prosocial interaction. In addition, we observed altered patterns of averaged and dynamic interbrain coherence in virtual interactions. Interbrain coherence patterns that were characteristic of the virtual condition were associated with reduced conversational turn-taking. These insights can inform the design and engineering of the next generation of videoconferencing technology.SIGNIFICANCE STATEMENTVideoconferencing has become an integral part of our lives. Whether this technology impacts behavior and neurobiology is not well understood. We explored potential effects of virtual interaction on social behavior, brain activity, and interbrain coupling. We found that virtual interactions were characterized by patterns of interbrain coupling that were negatively implicated in cooperation. Our findings are consistent with the perspective that videoconferencing technology adversely affects individuals and dyads during social interaction. As virtual interactions become even more necessary, improving the design of videoconferencing technology will be crucial for supporting effective communication.

Functional near-infrared spectroscopy (fNIRS) is a non-invasive neuroimaging technique with the potential to enable the assessment of posttraumatic stress disorder (PTSD) brain biomarkers in an affordable and portable manner. Consistent with biological models of PTSD, functional magnetic resonance imaging (fMRI) and fNIRS studies of adults with trauma exposure and PTSD symptoms suggest increased activation in the dorsolateral prefrontal cortex (dlPFC) and ventrolateral PFC (vlPFC) in response to negative emotion stimuli. We tested this theory with fNIRS assessment among youth exposed to traumatic stress and experiencing PTSD symptoms (PTSS). A portable fNIRS system collected hemodynamic responses from (N = 57) youth with PTSS when engaging in a classic emotion expression task that included fearful and neutral faces stimuli. The General Linear Model was applied to identify cortical activations associated with the facial stimuli. Subsequently, a prediction model was established via a Support Vector Regression to determine whether PTSS severity could be predicted based on fNIRS-derived cortical response measures and individual demographic information. Results were consistent with findings from adult fMRI and fNIRS studies of PTSS showing increased activation in the dlPFC and vlPFC in response to negative emotion stimuli. Subsequent prediction analysis revealed ten features (i.e., cortical responses from eight frontocortical fNIRS channels, age and sex) strongly correlated with PTSS severity (r = 0.65, p

Expressing appreciation is essential for establishing interpersonal closeness, but virtual interactions are increasingly common and create social distance. Little is known about the neural and inter-brain correlates of expressing appreciation and the potential effects of virtual videoconferencing on this kind of interaction. Here, we assess inter-brain coherence with functional near-infrared spectroscopy while dyads expressed appreciation to one another. We scanned 36 dyads (72 participants) who interacted in either an in-person or virtual (Zoom®) condition. Participants reported on their subjective experience of interpersonal closeness. As predicted, expressing appreciation increased interpersonal closeness between dyad partners. Relative to 3 other cooperation tasks (i.e. problem-solving task, creative-innovation task, socio-emotional task), we observed increased inter-brain coherence in socio-cognitive areas of the cortex (anterior frontopolar area, inferior frontal gyrus, premotor cortex, middle temporal gyrus, supramarginal gyrus, and visual association cortex) during the appreciation task. Increased inter-brain coherence in socio-cognitive areas during the appreciation task was associated with increased interpersonal closeness. These findings support the perspective that expressing appreciation, both in-person and virtually, increases subjective and neural metrics of interpersonal closeness.

BackgroundIdentifying neural activation patterns that predict youths’ treatment response may aid in the development of imaging-based assessment of emotion dysregulation following trauma and foster tailored intervention. Changes in cortical hemodynamic activity measured with functional near-infrared spectroscopy (fNIRS) may provide a time and cost-effective option for such work. We examined youths’ PTSD symptom change following treatment and tested if previously identified activation patterns would predict treatment response.MethodsYouth (N = 73, mean age = 12.97, SD = 3.09 years) were randomly assigned to trauma-focused cognitive behavioral therapy (TF-CBT), cue-centered therapy (CCT), or treatment as usual (TAU). Parents and youth reported on youth’s PTSD symptoms at pre-intervention, post-intervention, and follow-up. Neuroimaging data (N = 31) assessed at pre-intervention were obtained while youth engaged in an emotion expression task. Treatment response slopes were calculated for youth’s PTSD symptoms.ResultsOverall, PTSD symptoms decreased from pre-intervention through follow-up across conditions, with some evidence of relative benefit of TF-CBT and CCT over TAU but significant individual variation in treatment response. Cortical activation patterns were correlated with PTSD symptom improvement slopes (r = 0.53). In particular, cortical responses to fearful and neutral facial stimuli in six fNIRS channels in the bilateral dlPFC were important predictors of PTSD symptom improvement.ConclusionsThe use of fNIRS provides a method of monitoring and assessing cortical activation patterns in a relatively inexpensive and portable manner. Associations between functional activation and youths’ PTSD symptoms improvement may be a promising avenue for understanding emotion dysregulation in clinical populations.

Assessment of brain function with functional near-infrared spectroscopy (fNIRS) is limited to the outer regions of the cortex. Previously, we demonstrated the feasibility of inferring activity in subcortical “deep brain” regions using cortical fMRI and fNIRS activity in healthy adults. Access to subcortical regions subserving emotion and arousal using affordable and portable fNIRS is likely to be transformative for clinical diagnostic and treatment planning. Here, we validate the feasibility of inferring activity in subcortical regions that are central to the pathophysiology of PTSD (i.e., amygdala and hippocampus) using cortical fMRI and simulated fNIRS activity in a sample of adolescents diagnosed with PTSD (N=20, mean age=15.3±1.9 years) and age-matched healthy controls (N=20, mean age=14.5±2.0 years) as they performed a facial expression task. We tested different prediction models, including linear regression, a multi-layer perceptron neural network, and a k-nearest neighbors model. Inference of subcortical fMRI activity with cortical fMRI showed high prediction performance for the amygdala (r>0.91) and hippocampus (r>0.95) in both groups. Using fNIRS simulated data, relatively high prediction performance for deep brain regions was maintained in healthy controls (r>0.79), as well as in youths with PTSD (r>0.75). The linear regression and neural network models provided the best predictions.

The authors have withdrawn their manuscript because of a revised preprocessing pipeline. Therefore, the authors do not wish this work to be cited as reference for the project. If you have any questions, please contact the corresponding author.

In-car passive stress sensing could enable the monitoring of stress biomarkers while driving and reach millions of commuters daily (i.e., 123 million daily commuters in the US alone). Here, we present a nonintrusive method to detect stress solely from steering angle data of a regular car. The method uses inverse filtering to convert angular movement data into a biomechanical Mass Spring Damper model of the arm and extracts its damped natural frequency as an approximation of muscle stiffness, which in turn reflects stress. We ran a within-subject study (N = 22), in which commuters drove a vehicle around a closed circuit in both stress and calm conditions. As hypothesized, cohort analysis revealed a significantly higher damped natural frequency for the stress condition (P = .023, d = 0.723). Subsequent automation of the method achieved rapid (i.e., within 8 turns) stress detection in the individual with a detection accuracy of 77%.

How two brains communicate with each other during social interaction is highly dynamic and complex. Multi-person (i.e., hyperscanning) studies to date have focused on analyzing the entire time series of brain signals to reveal an overall pattern of inter-brain synchrony (IBS). However, this approach does not account for the dynamic nature of social interaction. In the present study, we propose a data-driven approach based on sliding windows and k-mean clustering to capture the dynamic modulation of IBS patterns during interactive cooperation tasks. We used a portable functional near-infrared spectroscopy (fNIRS) system to measure brain hemodynamic response between interacting partners (20 dyads) engaged in a creative design task and a 3D model building task. Results indicated that inter-personal communication during naturalistic cooperation generally presented with a series of dynamic IBS states along the tasks. Compared to the model building task, the creative design task appeared to involve more complex and active IBS between multiple regions in specific dynamic IBS states. In summary, the proposed approach stands as a promising tool to distill complex inter-brain dynamics associated with social interaction into a set of representative brain states with more fine-grained temporal resolution. This approach holds promise for advancing our current understanding of the dynamic nature of neurocognitive processes underlying social interaction.

In this book chapter, we present our scientific approach for applying the methods of fNIRS hyperscanning to decode distinct qualities of team interaction. Specifically, we are interested in detecting states of inter-brain synchrony that correlate with the behavioral states of cooperation and collaboration—terminologies which have been previously introduced as separate states in design thinking literature. We propose that the differentiation between those two concepts holds great promise for a better classification of team interaction, and a more thorough understanding of the dynamics leading to improved performance and (design) results. It is our hope that this work will provide more accurate and valuable information on human social interaction within working teams in the design thinking and related areas.

The ubiquity of technology in today’s world is exemplified by our ability to connect with each other instantly all around the globe. Advances in video conferencing capabilities combined with dramatic socio-dynamic shifts brought about by COVID-19 have redefined the ways in which humans interact in modern society. Human reliance on effective virtual interfacing (e.g., zoom conferencing) is evermore present in today’s COVID-19 world and will undoubtedly expand in the future. This unprecedented rise in digitalization has direct implications on the output and productivity of human interactions across all design (thinking) activities and practices. Working in a virtual environment limits access to traditional design thinking tools such as (analog) “artifacts” or “manipulatives” (e.g., physical prototypes, post-its, etc.). As both neuroscientists and design researchers, we are interested in elucidating the neurobiological signatures that underlie these adapted human-to-human interactions. Our overarching goal is to understand and uncover the differences in collaborative outcomes (e.g., creativity) and inter-brain synchrony in virtual versus in-person interactions using both analog and digital manipulatives. We proposed an emergent technology in brain-imaging—hyperscanning (i.e., measuring two brains simultaneously to derive measures of inter-brain synchrony) with functional near-infrared spectroscopy (fNIRS)—as an ideal brain-imaging technique to tackle this challenge. A better understanding of how the nuances of these dynamics impact inter-brain synchrony during an innovation event will provide new insights for interventions or technology that can help optimize successful interaction in both scenarios. To inform the design of future fNIRS hyperscanning studies, we review the existing fNIRS hyperscanning literature in this book chapter. On the basis of the existing literature, we highlight the current gaps in research regarding virtual interactions. We provide insight into current hurdles regarding fNIRS hyperscanning hardware and methodology and give recommendations on how to advance the field of fNIRS hyperscanning relevant to design research in the digital age.

Many producers of automated vehicle systems have begun testing autonomous vehicles on the road. In order to ensure safety and prevent crashes, human drivers are enlisted to monitor autonomous vehicles. However, operators of autonomous systems exhibit negative behavior adaptations in response to prolonged supervision of automation. To prevent the onset of undesirable behaviors in safety drivers, we must investigate driver state and behavior changes during the operation of highly automated vehicles. In the study presented here, we examine the effects of theoretical and practical training on the drivers' response to potentially critical situations in a longitudinal driving simulator study. We also present the effects of encountering a failure of the automated vehicle on driver state and behavior. We conducted a two-part panel driving simulator study (N=28), with an interval of 20–30 days between the training and testing sessions. We found that while participants with training are better prepared for a potential failure of the automation, participants in both conditions show a rise in sleepy or drowsy behavior before a potential failure of automation.

The understanding of stress and its impact on human performance is crucial to mitigate human error in the face of a threat. This is especially the case for critical incidents on a ship bridge, where human error can easily lead to severe danger for crew, cargo, and other vessels. To overcome the current limitations of robust objective stress measures that reliably detect (di-)stress under highly noisy conditions, we set out to explore whether salivary cortisol – the stress biomarker in medicine and psychology – is a valuable complementary assessment tool in a high-stress/emergency context. In a controlled within-subjects experiment (N = 12) using a ship bridge simulator, we measured stress levels under three conditions (80 min each): baseline, low stress (open water navigation task in autopilot), and high stress (open water emergency scenario). We sampled salivary cortisol at 10 min intervals in conjunction with heart rate (variability) monitoring, and subjective stress assessments from both participants and expert evaluators. Results validate salivary cortisol as a successful tool for detecting distress. Unlike the other stress measures, salivary cortisol strongly correlated with expert stress assessments (r = 0.856) and overt stress behavior like instances of freezing and missing response cues. Surprisingly, data further revealed decreased salivary cortisol across periods of self-assessed improved performance (i.e., eustress). In fact, data suggests an inverted u-relationship between performance and salivary cortisol. The findings have direct implications for the vast field of emergency training, and serve as a first important validation and benchmark to proceed with real life applications. © 2020 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).