Search

Traffic fatalities remain among the leading death causes worldwide. To reduce this figure, car safety is listed as one of the most important factors. To actively support human drivers, it is essential for advanced driving assistance systems to be able to recognize the driver's actions and intentions. Prior studies have demonstrated various approaches to recognize driving actions and intentions based on in-cabin and external video footage. Given the performance of self-supervised video pre-trained (SSVP) Video Masked Autoencoders (VMAEs) on multiple action recognition datasets, we evaluate the performance of SSVP VMAEs on the Honda Research Institute Driving Dataset for driver action recognition (DAR) and on the Brain4Cars dataset for driver intention recognition (DIR). Besides the performance, the application of an artificial intelligence system in a safety-critical environment must be capable to express when it is uncertain about the produced results. Therefore, we also analyze uncertainty estimations produced by a Bayes-by-Backprop last-layer (BBB-LL) and Monte-Carlo (MC) dropout variants of an VMAE. Our experiments show that an VMAE achieves a higher overall performance for both offline DAR and end-to-end DIR compared to the state-of-the-art. The analysis of the BBB-LL and MC dropout models show higher uncertainty estimates for incorrectly classified test instances compared to correctly predicted test instances.

Driver intention recognition (DIR) methods mostly rely on deep neural networks (DNNs). To use DNNs in asafety-critical real-world environment it is essential to quantify how confident the model is about the producedpredictions. Therefore, this study evaluates the performance and calibration of a temporal convolutionalnetwork (TCN) for multiple probabilistic deep learning (PDL) methods (Bayes-by-Backprop, Monte-Carlodropout, Deep ensembles, Stochastic Weight averaging - Gaussian, Multi SWA-G, cyclic Stochastic GradientHamiltonian Monte Carlo). Notably, we formalize an approach that combines optimization-based pre-trainingwith Hamiltonian Monte-Carlo (PT-HMC) sampling, aiming to leverage the strengths of both techniques. Ouranalysis, conducted on two pre-processed open-source DIR datasets, reveals that PT-HMC not only matchesbut occasionally surpasses the performance of existing PDL methods. One of the remaining challenges thatprohibits the integration of a PDL-based DIR system into an actual car is the computational requirements toperform inference. Therefore, future work could focus on optimizing PDL methods to be more computationallyefficient without sacrificing performance or the ability to estimate uncertainties.

Real-world applications of artificial intelligence that can potentially harm human beings should be able to express uncertainty about the made predictions. Probabilistic deep learning (DL) methods (e.g., variational inference [VI], VI last layer [VI-LL], Monte-Carlo [MC] dropout, stochastic weight averaging - Gaussian [SWA-G], and deep ensembles) can produce a predictive uncertainty but require expensive MC sampling techniques. Therefore, we evaluated if the probabilistic DL methods are uncertain when making incorrect predictions for an open-source driver intention recognition dataset and if a surrogate DL model can reproduce the uncertainty estimates. We found that all probabilistic DL methods are significantly more uncertain when making incorrect predictions at test time, but there are still instances where the models are very certain but completely incorrect. The surrogate DL models trained on the MC dropout and VI uncertainty estimates were capable of reproducing a significantly higher uncertainty estimate when making incorrect predictions., CC BY-NC-SA 4.0CORRESPONDING AUTHOR: K. VELLENGA (e-mail: koen.vellenga@volvocars.com)This work was supported by the Intention Recognition in Real Time for Automotive 3D Situation Awareness (IRRA) Project (https://www.vinnova.se/p/intention-recognition-i-realtid-for-automotive-3d-situation-awareness-irra/)., Intention recognition for real-time automotive 3D situation awareness

A team of fighter pilots in a distributed environment with limited access to information rely on technology to pursue teamwork. In order to design systems that support distributed teamwork, it is, therefore, necessary to understand how access to information affects the team members. Certain factors, such as mutual performance monitoring, shared mental models, adaptability, and backup behavior are considered essential for effective teamwork. We investigate these factors in this work, focusing on how visually communicated information affects fighter pilots’ perception of these factors. For that, a questionnaire including the teamwork factors in relation to certain defined scenarios that contain various levels of information was distributed to fighter pilots. We show that the studied factors are affected by the level of information available to the pilots. Especially, mutual performance monitoring increases with the degree of available information.

Every year worldwide more than one million people die and a further 50 million people are injured in traffic accidents. Therefore, the development of car safety features that actively support the driver in preventing accidents, is of utmost importance to reduce the number of injuries and fatalities. However, to establish this support it is necessary that the advanced driver assistance system (ADAS) understands the driver’s intended behavior in advance. The growing variety of sensors available for vehicles together with improved computer vision techniques, hence led to increased research directed towards inferring the driver’s intentions. This article reviews 64 driver intention recognition studies with regard to the maneuvers considered, the driving features included, the AI methods utilized, the achieved performance within the presented experiments, and the open challenges identified by the respected researchers. The article provides a high level analysis of the current technology readiness level of driver intention recognition technology to address the challenges to enable reliable driver intention recognition, such as the system architecture, implementation, and the purpose of the technology., CC BY-NC-ND 4.0CORRESPONDING AUTHOR: K. VELLENGA (e-mail: koen.vellenga@volvocars.com)This work was supported by the Intention Recognition in Real Time for Automotive 3D Situation Awareness (IRRA) Project (https://www.vinnova.se/p/intention-recognition-i-realtid-for-automotive-3d-situation-awareness-irra/)., Intention recognition for real-time automotive 3D situation awareness

Artificial intelligence (AI) is nowadays included into an increasing number of critical systems. Inclusion of AI in such systems may, however, pose a risk, since it is, still, infeasible to build AI systems that know how to function well in situations that differ greatly from what the AI has seen before. Therefore, it is crucial that future AI systems have the ability to not only function well in known domains, but also understand and show when they are uncertain when facing something unknown. In this paper, we evaluate four different methods that have been proposed to correctly quantifying uncertainty when the AI model is faced with new samples. We investigate the behaviour of these models when they are applied to samples far from what these models have seen before, and if they correctly attribute those samples with high uncertainty. We also examine if incorrectly classified samples are attributed with an higher uncertainty than correctly classified samples. The major finding from this simple experiment is, surprisingly, that the evaluated methods capture the uncertainty differently and the correlation between the quantified uncertainty of the models is low. This inconsistency is something that needs to be further understood and solved before AI can be used in critical applications in a trustworthy and safe manner. © 2020, Springer Nature Switzerland AG.

This paper demonstrates how to explore and visualize different types of structure in data, including clusters, anomalies, causal relations, and higher order relations. The methods are developed with the goal of being as automatic as possible and applicable to massive, streaming, and distributed data. Finally, a decentralized learning scheme is discussed, enabling finding structure in the data without collecting the data centrally., BIDAF

The execution of teamwork varies widely depending on the domain and task in question. Despite the considerable diversity of teams and their operation, researchers tend to aim for unified theories and models regardless of field. However, we argue that there is a need for translation and adaptation of the theoretical models to each specific domain. To this end, a case study was carried out on fighter pilots and it was investigated how teamwork is performed in this specialised and challenging environment, with a specific focus on the dependence on technology for these teams. The collaboration between the fighter pilots is described and analysed using a generic theoretical model for effective teamwork from the literature. The results show that domain-specific application and modification is needed in order for the model to capture fighter pilot's teamwork. The study provides deeper understanding of the working conditions for teams of pilots and gives design implications for how tactical support systems can enhance teamwork in the domain. Practitioner summary: This article presents a qualitative interview study with fighter pilots based on a generic theoretical teamwork model applied to the fighter domain. The purpose is to understand the conditions under which teams of fighter pilots work and to provide guidance for the design of future technological aids.

We describe a research process where fighter pilots’ behaviors were investigated from a teamwork perspective and the findings conveyed to the designers of cockpit interfaces in order to improve the fighter aircraft system. The teamwork perspective was selected because fighter aircraft are complex systems that require an advanced and trained pilot, who also, in addition to managing the aircraft systems needs to be a team player, collaborating with team members during dynamic and fast-paced circumstances to achieve the mission goals. A generic theoretical model for effective teamwork was selected as a starting point and a survey was conducted in order to investigate how fighter pilots collaborate during missions. The teamwork model and the survey results were then presented at workshops with designers of cockpit interfaces participating. The focus on the workshops was pilot teamwork and several design ideas aiming at improving the system for collaboration were generated.

In medicinal chemistry programs it is key to design and make compounds that are efficacious and safe. This is a long, complex, and difficult multiparameter optimization process, often including several properties with orthogonal trends. New methods for the automated design of compounds against profiles of multiple properties are thus of great value. Here we present a fragment-based reinforcement learning approach based on an actor-critic model, for the generation of novel molecules with optimal properties. The actor and the critic are both modeled with bidirectional long short-term memory (LSTM) networks. The AI method learns how to generate new compounds with desired properties by starting from an initial set of lead molecules and then improving these by replacing some of their fragments. A balanced binary tree based on the similarity of fragments is used in the generative process to bias the output toward structurally similar molecules. The method is demonstrated by a case study showing that 93% of the generated molecules are chemically valid and more than a third satisfy the targeted objectives, while there were none in the initial set.

We present a flexible deep convolutional neural network method for the analyse of arbitrary sized graph structures representing molecules. The method makes use of RDKit, an open-source cheminformatics software, allowing the incorporation of any global molecular (such as molecular charge) and local (such as atom type) information. We evaluate the method on the Side Effect Resource (SIDER) v4.1 dataset and show that it significantly outperforms another recently proposed method based on deep convolutional neural networks. We also reflect on how different types of information and input data affect the predictive power of our model. This reflection highlights several open problems that should be solved to further improve the use of deep learning within cheminformatics.

Data Science offers a set of powerful approaches for making new discoveries from large and complex data sets. It combines aspects of mathematics, statistics, machine learning, etc. to turn vast amounts of data into new insights and knowledge. However, the sole use of automatic data science techniques for large amounts of complex data limits the human user’s possibilities in the discovery process, since the user is estranged from the process of data exploration. This chapter describes the importance of Information Visualization (InfoVis) and visual analytics (VA) within data science and how interactive visualization can be used to support analysis and decision-making, empowering and complementing data science methods. Moreover, we review perceptual and cognitive aspects, together with design and evaluation methodologies for InfoVis and VA.

We present a flexible deep convolutional neural network method for the analysis of arbitrary sized graph structures representing molecules. This method, which makes use of the Lipinski RDKit module, an open-source cheminformatics software, enables the incorporation of any global molecular (such as molecular charge and molecular weight) and local (such as atom hybridization and bond orders) information. In this paper, we show that this method significantly outperforms another recently proposed method based on deep convolutional neural networks on several datasets that are studied. Several best practices for training deep convolutional neural networks on chemical datasets are also highlighted within the article, such as how to select the information to be included in the model, how to prevent overfitting and how unbalanced classes in the data can be handled.

We present a new self-organizing algorithm for classification of a data that combines and extends the strengths of several common machine learning algorithms, such as algorithms in self-organizing neural networks, ensemble methods and deep neural networks. The increased expression power is combined with the explanation power of self-organizing networks. Our algorithm outperforms both deep neural networks and ensembles of deep neural networks. For our evaluation case, we use production monitoring data from a complex steel manufacturing process, where data is both high-dimensional and has many nonlinear interdependencies. In addition to the improved prediction score, the algorithm offers a new deep-learning based approach for how computational resources can be focused in data exploration, since the algorithm points out areas of the input space that are more challenging to learn.

This paper presents findings about visual representations of cause and effect relationship's direction, strength, and uncertainty based on an online user study. While previous researches focus on accuracy and few attributes, our empirical user study examines accuracy and the subjective ratings on three different attributes of a cause and effect relationship edge. The cause and effect direction was depicted by arrows and tapered lines; causal strength by hue, width, and a numeric value; and certainty by granularity, brightness, fuzziness, and a numeric value. Our findings point out that both arrows and tapered cues work well to represent causal direction. Depictions with width showed higher conjunct accuracy and were more preferred than that with hue. Depictions with brightness and fuzziness showed higher accuracy and were marked more understandable than granularity. In general, depictions with hue and granularity performed less accurately and were not preferred compared to the ones with numbers or with width and brightness.

This paper investigates the nature of user participation in the process of designing fighter aircraft cockpits. The role of the users, i.e. pilots, in the design of cockpit interfaces is explored. We present the results of an on-line questionnaire with twelve designers of cockpit interfaces for fighter aircraft. The results show that the designers have highlighted the need for more opportunities to observe the pilots, and they wish to obtain more information and ideas from them. Moreover, a larger involvement from users as examiners and testers in the evaluation process was desirable. Access to users was considered unproblematic and the risk of misunderstandings was reported to be low. Moreover, the designers did not support the idea that users should design or take design decisions.

In the era of big data, it is imperative to assist the human analyst in the endeavor to find solutions to ill-defined problems, i.e. to “detect the expected and discover the unexpected” (Yi et al., 2008). To their aid, a plethora of analysis support systems is available to the analysts. However, these support systems often lack visual and interactive features, leaving the analysts with no opportunity to guide, influence and even understand the automatic reasoning performed and the data used. Yet, to be able to appropriately support the analysts in their sense-making process, we must look at this process more closely. In this paper, we present the results from interviews performed together with data analysts from the automotive industry where we have investigated how they handle the data, analyze it and make decisions based on the data, outlining directions for the development of analytical support systems within the area., BIDAF - A Big Data Analytics Framework for a Smart Society

Fighter pilots depend on collaboration and teamwork to perform successful air missions. However, such collaboration is challenging due to limitations in communication and the amount of data that can be shared between aircraft. In order to design future support systems for fighter pilots, this paper aims at characterizing how pilots collaborate while performing real-world missions. Our starting point is the “Big Five” model for effective teamwork, put forth by Salas et al. [1]. Fighter pilots were interviewed about their teamwork, and how they prepare and perform missions in teams. The results from the interviews were used to describe how pilots collaborate in teams, and to suggest relationships between the teamwork elements of the “Big Five” model for fighter pilots performing missions. The results presented in this paper are intended to inform designers and developers of cockpit displays, data links and decision support systems for fighter aircraft.

Fighter pilots performing air missions rely heavily on teamwork for successful outcomes. Designing systems that support such teamwork in highly dynamic missions is a challenging task, and to the best of our knowledge, current teamwork models are not specifically adapted for this domain. This paper presents a model of task performance for military fighter pilots based on the teamwork model “Big Five” proposed by Salas, Sims, and Burke [1]. The “Big Five” model consists of eight teamwork elements that are essential for successful team performance. In-depth interviews were performed with fighter pilots to explore and describe the teamwork elements for the fighter aircraft domain. The findings from these interviews are used to suggest where in the task cycle of mission performance each teamwork element comes in to play.

Effective team work is regarded as a key factor for success in missions performed by fighter aircraft in a Tactical Air Unit (TAU). Many factors contrib-ute to how a team will succeed in their mission. From the existing literature on teamwork, Salas, Sims and Burke [1], suggested five main factors and three sup-porting mechanisms for effective team work. These were proposed as the “Big Five” of teamwork. This article investigates if the model offered by Salas et al. is applicable to a TAU of fighter aircraft. Semi-structured interviews were carried out with six fighter pilots. The results of these interviews imply that the model has relevance for the teamwork in a TAU. Moreover, this paper discusses impli-cations for the design of future decision-support systems that support team effec-tiveness.

Information Fusion as an E-Service in Scholarly Information Use (INCITE)

With increasing numbers of scholarly publications, and multiplicity of publication-types and outlets, overviews of research fields have become a challenge. We bring together bibliometric methods, information retrieval, information fusion, and data visualization within a new project, INCITE - Information Fusion as an E-service in Scholarly Information Use, with the aim to develop improved methods and tools addressing emerging user-needs. In this paper we report on ongoing research within that project. (a) We elaborate on a qualitative user-study in which the emerging needs of researchers in the age of big data are explored. The study is based on interviews and dialogue with seven scholars at different academic levels. Data analysis was informed by adaptive theory, in accordance to which iterative pre-coding, provisional codes, and memo-writing were used to reach a more abstract level of analysis. A number of challenges related to the multiplicity of information sources and extent of data were identified including difficulties in keeping track of all the relevant sources; the inability to utilize extensive sets of data being taken for granted; and using data reduction strategies that at times go against the scholar’s own ideals of scholarly rigor. In analysing these difficulties, we have identified potential solutions that could facilitate the process of forming overviews of different research areas. (b) An example of such a solution is presented, which is builds on the Dempster-Shafer Theory and is designed to allow for interactive individual ranking of information sources in the process of a coordinated search across different information sources.

This paper discusses the results of four empirical evaluations that assess the effects that visualizing system metainformation have on decision-making, particularly on confidence, trust, workload, time and performance. These four case studies correspond to the analysis of (1) the effects that visualizing uncertainty associated with sensor values (position, speed, altitude, etc. and track quality) have on decision-making on a ground to air defense scenario; (2) the effects that the visualization of the car’s certainty on its own capability of driving autonomously have on drivers’ trust and performance; (3) the influence that the visualization of various qualifiers associated with the proposals given by the support system has on air traffic operators carrying out identification tasks and (4) the effects that the presentation of different abstraction levels of information have on classification tasks carried out by fighter pilots. We summarize the results of these four case studies and discuss lessons learned for the design of future computerized support systems regarding the visualization of meta-information., Presentation: http://www.cogsima2014.org/Program.html

We present an empirical study where the effects of three levels of system transparency of an automated target classification aid on fighter pilots’ performance and initial trust in the system were evaluated. The levels of transparency consisted of (1) only presenting text–based information regarding the specific object (without any automated support), (2) accompanying the text-based information with an automatically generated object class suggestion and (3) adding the incorporated sensor values with associated (uncertain) historic values in graphical form. The results show that the pilots needed more time to make a classification decision when being provided with display condition 2 and 3 than display condition 1. However, the number of correct classifications and the operators’ trust ratings were the highest when using display condition 3. No difference in the pilots’ decision confidence was found, yet slightly higher workload was reported when using display condition 3. The questionnaire results report on the pilots’ general opinion that an automatic classification aid would help them make better and more confident decisions faster, having trained with the system for a longer period.

Modern healthcare's need for knowledge sharing and bridging the research–practice gap requires new forms of collaboration, in which clinicians of varying clinical and research expertise work together over geographical and organisational borders. To support such distributed communities of practice (CoPs), an understanding of their collaboration processes, outcomes, challenges and enablers is needed. The article examines these issues through a case study of a long-running CoP, the Swedish Oral Medicine Network (SOMNet). SOMNet's main form of collaboration is monthly telephone conference meetings centred on case consultations. Cases are submitted by the clinicians via a Web-based system. The methods used were interviews, observations, and a questionnaire. The work adds to previous research by studying a distributed CoP explicitly focused on supporting the transfer of scientific results from researchers to practitioners. We found that the regular meetings give a rhythm to the community. The centrality of cases means an immediate benefit for the submitter while the community is provided an authentic context for learning. SOMNet yields opportunities for help and learning for diverse expertise levels; the type of benefits is affected by the participant's degree of oral medicine knowledge and collaboration involvement. There are challenges in accommodating varying levels of expertise and encouraging those less experienced to participate. Enablers of the collaboration include the participation of experts, meeting facilitators and well-adapted ITs.

To investigate the impact of visualizing car uncertainty on drivers' trust during an automated driving scenario, a simulator study was conducted. A between-group design experiment with 59 Swedish drivers was carried out where a continuous representation of the uncertainty of the car's ability to autonomously drive during snow conditions was displayed to one of the groups, whereas omitted for the control group. The results show that, on average, the group of drivers who were provided with the uncertainty representation took control of the car faster when needed, while they were, at the same time, the ones who spent more time looking at other things than on the road ahead. Thus, drivers provided with the uncertainty information could, to a higher degree, perform tasks other than driving without compromising with driving safety. The analysis of trust shows that the participants who were provided with the uncertainty information trusted the automated system less than those who did not receive such information, which indicates a more proper trust calibration than in the control group.

Threat evaluation (TE) is concerned with determining the intent, capability and opportunity of detected targets. To their aid, military operators use support systems that analyse incoming data and make inferences based on the active evaluation framework. Several interface and interaction guidelines have been proposed for the implementation of TE systems; however there is a lack of research regarding how to make these systems transparent to their operators. This paper presents the results from interviews conducted with TE operators focusing on the need for and possibilities of improving the transparency of TE systems through the visualization of uncertainty and the presentation of the system rationale. © 2013 Springer-Verlag Berlin Heidelberg.

In the defense domain, to estimate if a targetis threatening and to which degree is a complex task, thatis typically carried out by human operators due to the highrisks and uncertainties associated. To their aid, different supportsystems have been implemented to analyze the data and providerecommendations for actions. Since the ultimate responsibilitylies in human operators, it is of utmost importance that theytrust and know how to use these systems, as well as have anunderstanding of their inner workings, strengths and limitations.This paper presents, first, a formative user study to char-acterize how air traffic operators carry out threat evaluationrelated tasks. Grounded in these findings and in guidelinesfound in the literature, we present a transparent and highlyinteractive prototype that aims at reducing operator’s cognitiveload and support threat assessment activities. The literaturereview provided on design guidelines, the outcomes of the userstudy, the design of the prototype as well as the results of aninitial evaluation can provide guidance for both researchers andprospective developers of future threat evaluation systems.

Advancements in technology and the need for improving the pilots' working situations have stimulated the growth of automated functions within the fighter aircraft domain. Functions that aid the pilots perform their tasks and to make decisions fast in an often rapidly changing environment have been introduced with the ultimate aim of easing the pilots' workloads and improving their situation awareness. However, both positive and negative effects of automating pilots' tasks have been documented, such as decreased workload and skill degradation, and it has been acknowledged that it is important to consider the needs of the human operator when designing automated functions. Human-Centred Automation (HCA) has been suggested as an approach for designing automated systems that brings forward the positive effects of automation. However, how to apply the concept of HCA in light of the specific prerequisites and demands of the modern fighter aircraft domain has not been fully investigated. This paper presents the results from interviews made together with fighter aircraft system developers, where the concept of HCA was investigated. Their opinions of the applicability of the concept of HCA within the fighter aircraft domain is represented as well as their thoughts about identified HCA guidelines in relation to the proposed support system. Conclusions drawn from the study are that HCA is indeed an important concept to consider within the domain as well as that the proposed guidelines could be of great use during the design process. Additionallly, insight into how to design automated support systems with the concept of HCA in mind is given.

Automated detection of anomalous trajectories is an important problem in the surveillance domain. Various algorithms based on learning of normal trajectory patterns have been proposed for this problem. Yet, these algorithms suffer from one or more of the following limitations: First, they are essentially designed for offline anomaly detection in databases. Second, they are insensitive to local sub-trajectory anomalies. Third, they involve tuning of many parameters and may suffer from high false alarm rates. The main contribution of this paper is the proposal and discussion of the Sliding Window Local Outlier Conformal Anomaly Detector (SWLO-CAD), which is an algorithm for online detection of local sub-trajectory anomalies. It is an instance of the previously proposed Conformal anomaly detector and, hence, operates online with well-calibrated false alarm rate. Moreover, SWLO-CAD is based on Local outlier factor, which is a previously proposed outlier measure that is sensitive to local anomalies. Thus, SWLO-CAD has a unique set of properties that address the issues above.

The surveillance of large sea, air or land areas normally involves the analysis of large volumes of heterogeneous data from multiple sources. Timely detection and identification of anomalous behavior or any threat activity is an important objective for enabling homeland security. While it is worth acknowledging that many existing mining applications support identification of anomalous behavior, autonomous anomaly detection systems for area surveillance are rarely used in the real world. We argue that such capabilities and applications present two critical challenges: (1) they need to provide adequate user support and (2) they need to involve the user in the underlying detection process. In order to encourage the use of anomaly detection capabilities in surveillance systems, this paper analyzes the challenges that existing anomaly detection and behavioral analysis approaches present regarding their use and maintenance by users. We analyze input parameters, detection process, model representation and outcomes. We discuss the role of visualization and interaction in the anomaly detection process. Practical examples from our current research within the maritime domain illustrate key aspects presented.