Search

The quickly rising development of autonomous vehicle technology and increase of (semi-) autonomous vehicles on the road leads to an increased demand for more sophisticated human–machine-cooperation approaches to improve trust and acceptance of these new systems. In this work, we investigate the feeling of discomfort of human passengers while driving autonomously and the automatic detection of this discomfort with several model approaches, using the combination of different data sources. Based on a driving simulator study, we analyzed the discomfort reports of 50 participants for autonomous inner city driving. We found that perceived discomfort depends on the driving scenario (with discomfort generally peaking in complex situations) and on the passenger (resulting in interindividual differences in reported discomfort extend and duration). Further, we describe three different model approaches on how to predict the passenger discomfort using data from the vehicle’s sensors as well as physiological and behavioral data from the passenger. The model’s precision varies greatly across the approaches, the best approach having a precision of up to 80%. All of our presented model approaches use combinations of linear models and are thus fast, transparent, and safe. Lastly, we analyzed these models using the SHAP method, which enables explaining the models’ discomfort predictions. These explanations are used to infer the importance of our collected features and to create a scenario-based discomfort analysis. Our work demonstrates a novel approach on passenger state modelling with simple, safe, and transparent models and with explainable model predictions, which can be used to adapt the vehicles’ actions to the needs of the passenger.

Traffic situations like turning at intersections are destined for safety-critical situations and accidents. Human errors are one of the main reasons for accidents in these situations. A model that recognizes the driver's turning intent could help to reduce accidents by warning the driver or stopping the vehicle before a dangerous turning maneuver. Most models that aim at predicting the probability of a driver's turning intent use only contextual information, such as gap size or waiting time. The objective of this study is to investigate whether the combination of context information and brain activation measurements enhances the recognition of turning intent. We conducted a driving simulator study while simultaneously measuring brain activation using high-density fNIRS. A neural network model for turning intent recognition was trained on the fNIRS and contextual data. The input variables were analyzed using SHAP (SHapley Additive exPlanations) feature importance analysis to show the positive effect of the inclusion of brain activation data. Both the model's evaluation and the feature importance analysis suggest that the combination of context information and brain activation leads to an improved turning intent recognition. The fNIRS results showed increased brain activation differences during the “turn” decision-making phase before turning execution in parts of the left motor cortices, such as the primary motor cortex (PMC; putative BA 4), premotor area (PMA; putative BA 6), and supplementary motor area (SMA; putative BA 8). Furthermore, we also observed increased activation differences in the left prefrontal areas, potentially in the left middle frontal gyrus (putative BA 9), which has been associated with the control of executive functions, such as decision-making and action planning. We hypothesize that brain activation measurements could be a more direct indicator with potentially high specificity for the turning behavior and thus help to increase the recognition model's performance.

Some studies provide evidence that humans could actively exploit the alleged technological advantages of autonomous vehicles (AVs). This implies that humans may tend to interact differently with AVs as compared to human driven vehicles (HVs) with the knowledge that AVs are programmed to be risk-averse. Hence, it is important to investigate how humans interact with AVs in complex traffic situations. Here, we investigated whether participants would value interactions with AVs differently compared to HVs, and if these differences can be characterized on the behavioral and brain-level. We presented participants with a cover story while recording whole-head brain activity using fNIRS that they were driving under time pressure through urban traffic in the presence of other HVs and AVs. Moreover, the AVs were programmed defensively to avoid collisions and had faster braking reaction times than HVs. Participants would receive a monetary reward if they managed to finish the driving block within a given time-limit without risky driving maneuvers. During the drive, participants were repeatedly confronted with left-lane turning situations at unsignalized intersections. They had to stop and find a gap to turn in front of an oncoming stream of vehicles consisting of HVs and AVs. While the behavioral results did not show any significant difference between the safety margin used during the turning maneuvers with respect to AVs or HVs, participants tended to be more certain in their decision-making process while turning in front of AVs as reflected by the smaller variance in the gap size acceptance as compared to HVs. Importantly, using a multivariate logistic regression approach, we were able to predict whether the participants decided to turn in front of HVs or AVs from whole-head fNIRS in the decision-making phase for every participant (mean accuracy = 67.2%, SD = 5%). Channel-wise univariate fNIRS analysis revealed increased brain activation differences for turning in front of AVs compared to HVs in brain areas that represent the valuation of actions taken during decision-making. The insights provided here may be useful for the development of control systems to assess interactions in future mixed traffic environments involving AVs and HVs.

Causal Models are increasingly suggested as a means to reason about the behavior of cyber-physical systems in socio-technical contexts. They allow us to analyze courses of events and reason about possible alternatives. Until now, however, such reasoning is confined to the technical domain and limited to single systems or at most groups of systems. The humans that are an integral part of any such socio-technical system are usually ignored or dealt with by "expert judgment". We show how a technical causal model can be extended with models of human behavior to cover the complexity and interplay between humans and technical systems. This integrated socio-technical causal model can then be used to reason not only about actions and decisions taken by the machine, but also about those taken by humans interacting with the system. In this paper we demonstrate the feasibility of merging causal models about machines with causal models about humans and illustrate the usefulness of this approach with a highly automated vehicle example.