Search

In the past, research, which addressed driver behaviour in dilemma zone while approaching signalised intersections, did not focus on cognitive processes underlying stop/go decisions and collision risk after yellow-onset. Therefore, a study in a dynamic driving simulator with 20 participants was conducted to examine cognitive processes as basis of decision-making at signalised intersections. The study followed a 2×2 design with repeated measures. First, cognitive processes were triggered by perceptual cues, like the onset of yellow and a car in a leading position. Second, a cognitive distracting secondary task, the drivers had to solve interfered with cognitive processes. The results show, that a car in a leading position increases the probability not to stop after yellow-onset. Furthermore, the cognitive distracting secondary task leads to longer perception-response times (PRT) after yellow-onset, but only if there was no car in a leading position. Additionally, pupillary responses of the drivers during cognitive loading driving conditions are supporting this pattern of results. Finally, the concept of coupled motion is suggested to explain the underlying stop/go pattern of drivers after yellow-onset.

Future automated vehicles (AVs) of different sizes will share the same space with other road users, e. g., pedestrians. For a safe interaction, successful communication needs to be ensured, in particular, with vulnerable road users, such as pedestrians. Two possible communication means exist for AVs: vehicle kinematics for implicit communication and external human-machine interfaces (eHMIs) for explicit communication. However, the exact interplay is not sufficiently studied yet for pedestrians' interactions with AVs. Additionally, very few other studies focused on the interplay of vehicle kinematics and eHMI for pedestrians' interaction with differently sized AVs, although the precise coordination is decisive to support the communication with pedestrians. Therefore, this study focused on how the interplay of vehicle kinematics and eHMI affects pedestrians' willingness to cross, trust and perceived safety for the interaction with two differently sized AVs (smaller AV vs. larger AV). In this experimental online study (N = 149), the participants interacted with the AVs in a shared space. Both AVs were equipped with a 360° LED light-band eHMI attached to the outer vehicle body. Three eHMI statuses (no eHMI, static eHMI, and dynamic eHMI) were displayed. The vehicle kinematics were varied at two levels (non-yielding vs. yielding). Moreover, “non-matching” conditions were included for both AVs in which the dynamic eHMI falsely communicated a yielding intent although the vehicle did not yield. Overall, results showed that pedestrians' willingness to cross was significantly higher for the smaller AV compared to the larger AV. Regarding the interplay of vehicle kinematics and eHMI, results indicated that a dynamic eHMI increased pedestrians' perceived safety when the vehicle yielded. When the vehicle did not yield, pedestrians' perceived safety still increased for the dynamic eHMI compared to the static eHMI and no eHMI. The findings of this study demonstrated possible negative effects of eHMIs when they did not match the vehicle kinematics. Further implications for a holistic communication strategy for differently sized AVs will be discussed.

Abstract Introduction Designing emotion-aware systems has become a manageable aim through recent developments in computer vision and machine learning. In the context of driver behaviour, especially negative emotions like frustration have shifted into the focus of major car manufacturers. Recognition and mitigation of the same could lead to safer roads in manual and more comfort in automated driving. While frustration recognition and also general mitigation methods have been previously researched, the knowledge of reasons for frustration is necessary to offer targeted solutions for frustration mitigation. However, up to the present day, systematic investigations about reasons for frustration behind the wheel are lacking. Methods Therefore, in this work a combination of diary study and user focus groups was employed to shed light on reasons why humans become frustrated during driving. In addition, participants of the focus groups were asked for their usual coping methods with frustrating situations. Results It was revealed that the main reasons for frustration in driving are related to traffic, in-car reasons, self-inflicted causes, and weather. Coping strategies that drivers use in everyday life include cursing, distraction by media and thinking about something else, amongst others. This knowledge will help to design a frustration-aware system that monitors the driver’s environment according to the spectrum of frustration causes found in the research presented here.

Facial expressions are one of the commonly used implicit measurements for the in-vehicle affective computing. However, the time courses and the underlying mechanism of facial expressions so far have been barely focused on. According to the Component Process Model of emotions, facial expressions are the result of an individual's appraisals, which are supposed to happen in sequence. Therefore, a multidimensional and dynamic analysis of drivers' fear by using facial expression data could profit from a consideration of these appraisals. A driving simulator experiment with 37 participants was conducted, in which fear and relaxation were induced. It was found that the facial expression indicators of high novelty and low power appraisals were significantly activated after a fear event (high novelty: Z = 2.80, p < 0.01, rcontrast = 0.46; low power: Z = 2.43, p < 0.05, rcontrast = 0.50). Furthermore, after the fear event, the activation of high novelty occurred earlier than low power. These results suggest that multidimensional analysis of facial expression is suitable as an approach for the in-vehicle measurement of the drivers' emotions. Furthermore, a dynamic analysis of drivers' facial expressions considering of effects of appraisal components can add valuable information for the in-vehicle assessment of emotions.

Driving is a complex task concurrently drawing on multiple cognitive resources. Yet, there is a lack of studies investigating interactions at the brain-level among different driving subtasks in dual-tasking. This study investigates how visuospatial attentional demands related to increased driving difficulty interacts with different working memory load (WML) levels at the brain level. Using multichannel whole-head high density functional near-infrared spectroscopy (fNIRS) brain activation measurements, we aimed to predict driving difficulty level, both separate for each WML level and with a combined model. Participants drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. In half of the time, the course led through a construction site with reduced lane width, increasing visuospatial attentional demands. Concurrently, participants performed a modified version of the n-back task with five different WML levels (from 0-back up to 4-back), forcing them to continuously update, memorize, and recall the sequence of the previous ‘n’ speed signs and adjust their speed accordingly. Using multivariate logistic ridge regression, we were able to correctly predict driving difficulty in 75.0% of the signal samples (1.955 Hz sampling rate) across 15 participants in an out-of-sample cross-validation of classifiers trained on fNIRS data separately for each WML level. There was a significant effect of the WML level on the driving difficulty prediction accuracies [range 62.2–87.1%; χ2(4) = 19.9, p < 0.001, Kruskal–Wallis H test] with highest prediction rates at intermediate WML levels. On the contrary, training one classifier on fNIRS data across all WML levels severely degraded prediction performance (mean accuracy of 46.8%). Activation changes in the bilateral dorsal frontal (putative BA46), bilateral inferior parietal (putative BA39), and left superior parietal (putative BA7) areas were most predictive to increased driving difficulty. These discriminative patterns diminished at higher WML levels indicating that visuospatial attentional demands and WML involve interacting underlying brain processes. The changing pattern of driving difficulty related brain areas across WML levels could indicate potential changes in the multitasking strategy with level of WML demand, in line with the multiple resource theory.

Experiencing frustration while driving can harm cognitive processing, result in aggressive behavior and hence negatively influence driving performance and traffic safety. Being able to automatically detect frustration would allow adaptive driver assistance and automation systems to adequately react to a driver’s frustration and mitigate potential negative consequences. To identify reliable and valid indicators of driver’s frustration, we conducted two driving simulator experiments. In the first experiment, we aimed to reveal facial expressions that indicate frustration in continuous video recordings of the driver’s face taken while driving highly realistic simulator scenarios in which frustrated or non-frustrated emotional states were experienced. An automated analysis of facial expressions combined with multivariate logistic regression classification revealed that frustrated time intervals can be discriminated from non-frustrated ones with accuracy of 62.0% (mean over 30 participants). A further analysis of the facial expressions revealed that frustrated drivers tend to activate muscles in the mouth region (chin raiser, lip pucker, lip pressor). In the second experiment, we measured cortical activation with almost whole-head functional near-infrared spectroscopy (fNIRS) while participants experienced frustrating and non-frustrating driving simulator scenarios. Multivariate logistic regression applied to the fNIRS measurements allowed us to discriminate between frustrated and non-frustrated driving intervals with higher accuracy of 78.1% (mean over 12 participants). Frustrated driving intervals were indicated by increased activation in the inferior frontal, putative premotor and occipito-temporal cortices. Our results show that facial and cortical markers of frustration can be informative for time resolved driver state identification in complex realistic driving situations. The markers derived here can potentially be used as an input for future adaptive driver assistance and automation systems that detect driver frustration and adaptively react to mitigate it.

Cognitive overload or underload results in a decrease in human performance which may result in fatal incidents while driving. We envision that driver assistive systems which adapt their functionality to the driver’s cognitive state could be a promising approach to reduce road accidents due to human errors. This research attempts to predict variations of cognitive working memory load levels in a natural driving scenario with multiple parallel tasks and to reveal predictive brain areas. We used a modified version of the n-back task to induce five different working memory load levels (from 0-back up to 4-back) forcing the participants to continuously update, memorize, and recall the previous ‘n’ speed sequences and adjust their speed accordingly while they drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. We measured brain activation using multichannel whole head, high density functional near-infrared spectroscopy (fNIRS) and predicted working memory load level from the fNIRS data by combining multivariate lasso regression and cross-validation. This allowed us to predict variations in working memory load in a continuous time-resolved manner with mean Pearson correlations between induced and predicted working memory load over 15 participants of 0.61 [standard error (SE) 0.04] and a maximum of 0.8. Restricting the analysis to prefrontal sensors placed over the forehead reduced the mean correlation to 0.38 (SE 0.04), indicating additional information gained through whole head coverage. Moreover, working memory load predictions derived from peripheral heart rate parameters achieved much lower correlations (mean 0.21, SE 0.1). Importantly, whole head fNIRS sampling revealed increasing brain activation in bilateral inferior frontal and bilateral temporo-occipital brain areas with increasing working memory load levels suggesting that these areas are specifically involved in workload-related processing.