Search

Adaptive visualization and interfaces pervade our everyday tasks to improve interaction from the point of view of user performance and experience. This approach allows using several user inputs, whether physiological, behavioral, qualitative, or multimodal combinations, to enhance the interaction. Due to the multitude of approaches, we outline the current research trends of inputs used to adapt visualizations and user interfaces. Moreover, we discuss methodological approaches used in mixed reality, physiological computing, visual analytics, and proficiency-aware systems. With this work, we provide an overview of the current research in adaptive systems.

Our work for the first time evaluates the effectiveness of visual and acoustic warning systems in an accident situation using a realistic, immersive driving simulation. In a first experiment, 70 participants were trained to complete a course at high speed. The course contained several forks where a wrong turn would lead to the car falling off a cliff and crashing - these forks were indicated either with a visual warning sign for a first, no-sound group or with a visual and auditory warning cue for a second, sound group. In a testing phase, right after the warning signals were given, trees suddenly fell on the road, leaving the (fatal) turn open. Importantly, in the no-sound group, 18 out of 35 people still chose this turn, whereas in the sound group only 5 out of 35 people did so - the added sound therefore had a large and significant increase in situational awareness. We found no other differences between the groups concerning age, physiological responses, or driving experience. In a second replication experiment, the setup was repeated with another 70 participants without emphasis on driving speed. Results fully confirmed the previous findings with 17 out of 35 people in the no-sound group versus only 6 out of 35 in the sound group choosing the turn to the cliff. With these two experiments using a one-shot design to avoid pre-meditation and testing naive, rapid decision-making, we provide clear evidence for the advantage of visual-auditory in-vehicle warning systems for promoting situational awareness.

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section “An empirically based minimal reporting guideline”).

Blinking is a natural user-induced response which paces visual information processing. This study investigates whether blinks are viable for segmenting continuous electroencephalography (EEG) activity, for inferring cognitive demands in ecologically valid work environments. We report the blink-related EEG measures of participants who performed auditory tasks either standing, walking on grass, or whilst completing an obstacle course. Blink-related EEG activity discriminated between different levels of cognitive demand during walking. Both behavioral parameters (e.g., blink duration or head motion) and blink-related EEG activity varied with walking conditions. Larger occipital N1 was observed during walking, relative to standing and traversing an obstacle course, which reflects differences in bottom-up visual perception. In contrast, the amplitudes of top-down components (N2, P3) significantly decreased with increasing walking demands, which reflected narrowing attention. This is consistent with blink-related EEG, specifically in Theta and Alpha power that, respectively, increased and decreased with increasing demands of the walking task. This work presents a novel and robust analytical approach to evaluate the cognitive demands experienced in natural work settings, which precludes the use of artificial task manipulations for data segmentation.

Background Mandatory rules for social distancing to curb the SARS-CoV-2 pandemic require individuals to maintain a critical interpersonal distance above 1.5 m. However, this contradicts our natural preference, which is closer to 1 m for non-intimate encounters, for example, when asking a stranger for directions. Objective This review addresses how humans typically regulate interpersonal distances, in order to highlight the challenges of enforcing atypically large interpersonal distances. Method To understand the challenges posed by social distancing requirements, we integrate relevant contributions from visual perception, social perception, and human factors. Results To date, research on preferred interpersonal distances suggests that social distancing could induce discomfort, heighten arousal, and decrease social signaling in the short term. While the protracted effects of social distancing are unclear, we propose hypotheses on the mid- to long-term consequences of violating preferred norms of interpersonal distances. Conclusion We suggest that enforcing a physical distance of 1.5–2 m presents a serious challenge to behavioral norms. Application We address how notifications, architectural design, and visualizations could be effectively applied to promote interpersonal distance requirements.

Consumer neurotechnology is arriving en masse, even while algorithms for user state estimation are being actively defned and developed. Indeed, many consumable wearables are now available that try to estimate cognitive changes from wrist data or body movement. But does this data help people? It’s a critical time to ask how users could be informed by wearable neurotechnology, in a way that would be relevant to their needs and serve their personal well-being. The aim of this SIG is to bring together the key HCI communities needed to address this: personal informatics, digital health and wellbeing, neuroergonomics, and neuroethics.

The technological properties of a system delivering simulation experience are a cru- cial dimension of immersion. To create a sense of presence and reproduce drivers behaviour as realistically as possible, we need reliable driving simulators that allow drivers to become highly immersed. This study investigates the impact of a system immersion of a driving simulator on the drivers’ brain activity while operating a con- ditionally automated vehicle. Nineteen participants drove approximately 40 minutes while their brain activity was recorded using electroencephalography (EEG). We found a significant effect of the system immersion in the occipital and parietal areas, prima- rily in the high-Beta bandwidth. No effect was found in the Theta, Alpha, and low-Beta bandwidths. These findings suggest that the system immersion might influence the drivers’ physiological arousal, consequently influencing their cognitive and emotional processes.

Manual control of robotic arms is challenging and productive operators require extensive prior training. Effective training should systematically vary the difficulty level of the robot arm motions. This study investigates the extent to which Fitts’ law could define movement difficulty for bimanual controlled movements of robotic arms. Inspired by forestry work-methods, we designed Fitts’ tapping Task to assess the movement time and throughput of ten unskilled participants over nine training sessions. We found that robotic arm movements observe Fitts’ law for reaching in depth but deviate for lateral and concentric movements. In other words, training can utilize Fitts’ law to vary the difficulty of forward robot arm movements. Further studies on the difficulty of lateral and concentric movements are necessary to refine work methods and improve training.

Detecting emotions while driving remains a challenge in Human-Computer Interaction. Current methods to estimate the driver’s experienced emotions use physiological sensing (e.g., skin-conductance, electroencephalography), speech, or facial expressions. However, drivers need to use wearable devices, perform explicit voice interaction, or require robust facial expressiveness. We present VEmotion (Virtual Emotion Sensor), a novel method to predict driver emotions in an unobtrusive way using contextual smartphone data. VEmotion analyzes information including traffic dynamics, environmental factors, in-vehicle context, and road characteristics to implicitly classify driver emotions. We demonstrate the applicability in a real-world driving study (N = 12) to evaluate the emotion prediction performance. Our results show that VEmotion outperforms facial expressions by 29% in a person-dependent classification and by 8.5% in a person-independent classification. We discuss how VEmotion enables empathic car interfaces to sense the driver’s emotions and will provide in-situ interface adaptations on-the-go.

The Interactions website (interactions.acm.org) hosts a stable of bloggers who share insights and observations on HCI, often challenging current practices. Each issue we'll publish selected posts from some of the leading and emerging voices in the field.

Even with highly automated driving, there will exist the occasional need for a human driver to resume control. The design of take-over requests (TOR) and the driving scenario upon their presentation are factors that can be expected to determine the effective resumption of vehicle control. We conducted a driving simulator study in the laboratory that examined how either an abstract or skeuomorphic TOR design influenced driving resumption, given different conditions of road curvature and time headway. We found that drivers react faster and more accurately when experiencing a skeuomorphic visualisation of a TOR prompt compared to an abstract interface, especially when there is less time headway. Moreover, participants reported subjective preferences for the skeuomorphic TOR design and considered them to be more intuitively comprehensible. Furthermore, more accurate TOR responses were found when TORs of lateral control were presented before longitudinal control. Based on these results, we formulated five conclusions relating to the design of take-over requests. These conclusions should be validated in more realistic settings. However, in the future, they could be the basis for more research developing design guidelines that could create safe and reliable communications between drivers and vehicles.

Interactive computing systems are able to receive, as inputs, activity generated by the user’s physiology (e.g., skin conductance, heart rate, brain potentials, and so forth). Besides health-related applications, this type of physiological sensing enables systems to infer users’ states (e.g., task engagement, anxiety, workload, and so forth). More recently, a number of techniques emerged that can also stimulate physiological activity (e.g., electrical muscle stimulation, galvanic vestibular stimulation, transcranial stimulation). These can serve as outputs of an interactive system to induce desired behavior in the user. Taken together, we envision systems that will close the loop between physiological input and output—interactive systems able to read and influence the user’s body. To realize this, we propose a Special Interest Group on Physiological I/O that will consolidate successful practices and identify research challenges to address as a community.

A lot of academic and industrial HCI work has focused on making interactions easier and less effortful. As the potential risks of optimising for effortlessness have crystallised in systems designed to take advantage of the way human attention and cognition works, academic researchers and industrial practitioners have wondered whether increasing the ‘friction’ in interactions, making them more effortful might make sense in some contexts. The goal of this special interest group is to provide a forum for researchers and practitioners to discuss and advance the theoretical underpinnings of designed friction, the relation of friction to other design paradigms, and to identify the domains and interaction flows that frictions might best suit. During the SIG, attendees will attempt to prioritise a set of research questions about frictions in HCI.

There is a growing interest in eye tracking technologies applied to support traditional visualization techniques like diagrams, charts, maps, or plots, either static, animated, or interactive ones. More complex data analyses are required to derive knowledge and meaning from the data. Eye tracking systems serve that purpose in combination with biological and computer vision, cognition, perception, visualization, human-computer-interaction, as well as usability and user experience research. The 10 articles collected in this thematic special issue provide interesting examples how sophisticated methods of data analysis and representation enable researchers to discover and describe fundamental spatio-temporal regularities in the data. The human visual system, supported by appropriate visualization tools, enables the human operator to solve complex tasks, like understanding and interpreting three-dimensional medical images, controlling air traffic by radar displays, supporting instrument flight tasks, or interacting with virtual realities. The development and application of new visualization techniques is of major importance for future technological progress.

While watching omnidirectional movies via head-mounted displays the viewer has an immersive viewing experience. Turning the head and looking around is a natural input technique to choose the visible part of the movie. For realizing scene changes depending on the viewing direction and for implementing non-linear story structures in cinematic virtual reality (CVR), selection methods are required to select the story branches. The input device should not disturb the viewing experience and the viewer should not be primarily aware of it. Eye- and head-based methods do not need additional devices and seem to be especially suitable. We investigate several techniques by using an own tool for analysing head and eye tracking data in CVR.

How are strong positive affective states related to eye-tracking features and how can they be used to appropriately enhance well-being in multimedia consumption? In this paper, we propose a robust classification algorithm for predicting strong happy emotions from a large set of features acquired from wearable eye-tracking glasses. We evaluate the potential transferability across subjects and provide a model-agnostic interpretable feature importance metric. Our proposed algorithm achieves a true-positive-rate of 70% while keeping a low false-positive-rate of 10% with extracted features of the pupil diameter as most important features.

Rapid Serial Visual Presentation (RSVP) has gained popularity as a method for presenting text on wearable devices with limited screen space. Nonetheless, it remains unclear how to calibrate RSVP display parameters, such as spatial alignments or presentation rates, to suit the reader's information processing ability at high presentation speeds. Existing methods rely on comprehension and subjective workload scores, which are influenced by the user's knowledge base and subjective perception. Here, we use electroencephalography (EEG) to directly determine how individual information processing varies with changes in RSVP display parameters. Eighteen participants read text excerpts with RSVP in a repeated-measures design that manipulated the Text Alignment and Presentation Speed of text representation. We evaluated how predictive EEG metrics were of gains in reading speed, subjective workload, and text comprehension. We found significant correlations between EEG and increasing Presentation Speeds and propose how EEG can be used for dynamic selection of RSVP parameters.

Manual assembly at production is a mentally demanding task. With rapid prototyping and smaller production lot sizes, this results in frequent changes of assembly instructions that have to be memorized by workers. Assistive systems compensate this increase in mental workload by providing "just-in-time" assembly instructions through in-situ projections. The implementation of such systems and their benefits to reducing mental workload have previously been justified with self-perceived ratings. However, there is no evidence by objective measures if mental workload is reduced by in-situ assistance. In our work, we showcase electroencephalography (EEG) as a complementary evaluation tool to assess cognitive workload placed by two different assistive systems in an assembly task, namely paper instructions and in-situ projections. We identified the individual EEG bandwidth that varied with changes in working memory load. We show, that changes in the EEG bandwidth are found between paper instructions and in-situ projections, indicating that they reduce working memory compared to paper instructions. Our work contributes by demonstrating how design claims of cognitive demand can be validated. Moreover, it directly evaluates the use of assistive systems for delivering context-aware information. We analyze the characteristics of EEG as real-time assessment for cognitive workload to provide insights regarding the mental demand placed by assistive systems.

Objective This study investigates the neural basis of inattentional deafness, which could result from task irrelevance in the auditory modality. Background Humans can fail to respond to auditory alarms under high workload situations. This failure, termed inattentional deafness, is often attributed to high workload in the visual modality, which reduces one's capacity for information processing. Besides this, our capacity for processing auditory information could also be selectively diminished if there is no obvious task relevance in the auditory channel. This could be another contributing factor given the rarity of auditory warnings. Method Forty-eight participants performed a visuomotor tracking task while auditory stimuli were presented: a frequent pure tone, an infrequent pure tone, and infrequent environmental sounds. Participants were required either to respond to the presentation of the infrequent pure tone (auditory task-relevant) or not (auditory task-irrelevant). We recorded and compared the event-related potentials (ERPs) that were generated by environmental sounds, which were always task-irrelevant for both groups. These ERPs served as an index for our participants' awareness of the task-irrelevant auditory scene. Results Manipulation of auditory task relevance influenced the brain's response to task-irrelevant environmental sounds. Specifically, the late novelty-P3 to irrelevant environmental sounds, which underlies working memory updating, was found to be selectively enhanced by auditory task relevance independent of visuomotor workload. Conclusion Task irrelevance in the auditory modality selectively reduces our brain's responses to unexpected and irrelevant sounds regardless of visuomotor workload. Application Presenting relevant auditory information more often could mitigate the risk of inattentional deafness.

This work presents a visual analytics approach to explore microsaccade distributions in high-frequency eye tracking data. Research studies often apply filter algorithms and parameter values for microsaccade detection. Even when the same algorithms are employed, different parameter values might be adopted across different studies. In this paper, we present a visual analytics system (VisME) to promote reproducibility in the data analysis of microsaccades. It allows users to interactively vary the parametric values for microsaccade filters and evaluate the resulting influence on microsaccade behavior across individuals and on a group level. In particular, we exploit brushing-and-linking techniques that allow the microsaccadic properties of space, time, and movement direction to be extracted, visualized, and compared across multiple views. We demonstrate in a case study the use of our visual analytics system on data sets collected from natural scene viewing and show in a qualitative usability study the usefulness of this approach for eye tracking researchers. We believe that interactive tools such as VisME will promote greater transparency in eye movement research by providing researchers with the ability to easily understand complex eye tracking data sets; such tools can also serve as teaching systems. VisME is provided as open source software.

Driver assistance system development commonly targets a substitution of driver responsibilities. Such a substitutive approach which ignores the potential of utilizing available human resources has at least two downsides: 1. The human mind has not evolved to stay idle such that people tend to engage in secondary tasks and disengage from the driving task when not needed, causing out-of-the-loop effects. 2. People have strengths in domains that have not yet been mastered artificially and which could substantially improve human-machine systems. This workshop will focus on a different approach that emphasizes on strengthening the link between vehicle and driver. Guided by the term "the embodied vehicle" we will draft and discuss options for achieving a closer human-vehicle integration and try to identify possible implications of the developed approaches. The workshop aims to connect practitioners, researchers, and professionals who wish to further explore this perspective on technology and interface development.

Driving simulators are necessary for evaluating automotive technology for human users. While they can vary in terms of their fidelity, it is essential that users experience minimal simulator sickness and high presence in them. In this paper, we present two experiments that investigate how a virtual driving simulation system could be visually presented within a real vehicle, which moves on a test track but displays a virtual environment. Specifically, we contrasted display presentation of the simulation using either head-mounted displays (HMDs) or fixed displays in the vehicle itself. Overall, we find that fixed displays induced less simulator sickness than HMDs. Neither HMDs or fixed displays induced a stronger presence in our implementation, even when the field-of-view of the fixed display was extended. We discuss the implications of this, particular in the context of scenarios that could induce considerable motion sickness, such as testing non-driving activities in automated vehicles.

High-stress work environments, such as a flight deck or surveillance systems, provide operators with multiple instruments that have to be monitored constantly using eye movements. Eye tracking allows us to infer when an operator’s overt attention is devoted to an instrument, namely when fixation begins. It is commonly assumed that fixating a visual stimulus equates its processing. In this study, we use an electroencephalogram (EEG) to demonstrate that visual perception, hence processing, can take place even prior to fixation. We show that target onsets give rise to brain responses that are associated with stimulus processing even before targets were fixated. This poses a challenge for the use of the electroencephalogram/event-related potential (EEG/ERP) in visual scanning environments: if fixation onset is not necessarily the onset of perception, it cannot always be used for attributing ERP data.

Auditory cues are often used to capture and direct attention, away from an ongoing task to a critical situation. In the context of driving, previous research have shown that looming sounds, which convey time-to-contact information through their rising-intensity profiles, promote faster braking times to potential front collisions. The current experiment investigates the role of auditory warnings in facilitating the identification of visual objects in the visual periphery during steering. This approximates the use of auditory warnings for cueing possible candidates for side collisions. We expected faster response times for visual targets cued by a looming sound compared to a constant sound. Electroencephalography (EEG) was recorded to determine whether faster response times were due to either earlier or stronger neural responses to the visual target. We hypothesize: (1) earlier event-related potentials (ERPs) for cued compared to non-cued visual targets, and (2) larger amplitudes for visual targets that were cued by looming versus constant sounds.

Looming sounds can be an ideal warning notification for emergency braking. This agrees with studies that have consistently demonstrated preferential brain processing for looming stimuli. This study investigates and demonstrates that looming sounds can similarly benefit emergency braking in managing a vehicle with adaptive cruise control (ACC). Specifically, looming auditory notifications induced the faster emergency braking times relative to a static auditory notification. Next, we compare the event-related potential (ERP) evoked by a looming notification, relative to its static equivalent. Looming notifications evoke a smaller fronto-central N2 amplitude than their static equivalents. Thus, we infer that looming sounds are consistent with the visual experience of an approaching collision and, hence, induced a corresponding performance benefit. Subjective ratings indicate no significant differences in the perceived workload across the notification conditions. Overall, this work suggests that auditory warnings should have congruent physical properties with the visual events that they warn for.

Currently offered autonomous vehicles still require the human intervention. For instance, when the system fails to perform as expected or adapts to unanticipated situations. Given that reliability of autonomous systems can fluctuate across conditions, this work is a first step towards understanding how this information ought to be communicated to users. We conducted a user study to investigate the effect of communicating the system's reliability through a feedback bar. Subjective feedback was solicited from participants with questionnaires and semi-structured interviews. Based on the qualitative results, we derived guidelines that serve as a foundation for the design of how autonomous systems could provide continuous feedback on their reliability.

Touchscreens are a common fixture in current vehicles. With autonomous driving, we can expect touch interaction with such in-vehicle media systems to exponentially increase. In spite of vehicle suspension systems, road perturbations will continue to exert forces that can render in-vehicle touch interaction challenging. Using a motion simulator, we investigate how different vehicle speeds interact with road features (i.e., speed bumps) to influence touch interaction. We determine their effect on pointing accuracy and task completion time. We show that road bumps have a significant effect on touch input and can decrease accuracy by 19%. In light of this, we developed a Random Forest (RF) model that improves touch accuracy by 32.0% on our test set and by 22.5% on our validation set. As the lightweight model uses only features that can easily be determined through inertial measurement units, this model could be easily deployed in current automobiles.

This workshop discusses the balance between safety and productivity as automated vehicles turn into 'mobile offices': spaces where non-driving activities are performed during one's daily commute. Technological developments reduce the active role of the human driver that might, nonetheless, require occasional intervention. To what extent are drivers allowed to dedicate resources to non-driving work-related activities? To address this critical question, the workshop brings together a diverse community of researchers and practitioners that are interested in questions as follows: what non-driving activities are likely to be performed on one's way to work and back; what is a useful taxonomy of these tasks; how can various tasks be studied in experimental settings; and, what are the criteria to assess human performance in automated vehicles. To foster further dialogue, the outcome of the workshop will be an online blog where attendees can contribute their own thoughts: https://medium.com/the-mobile-office.

There is a growing interest in eye tracking technologies applied to support traditional visualization techniques like diagrams, charts, maps, or plots, either static, animated, or interactive ones. More complex data analyses are required to derive knowledge and meaning from the data. Eye tracking systems serve that purpose in combination with biological and computer vision, cognition, perception, visualization, human-computer-interaction, as well as usability and user experience research. The 10 articles collected in this thematic special issue provide interesting examples how sophisticated methods of data analysis and representation enable researchers to discover and describe fundamental spatio-temporal regularities in the data. The human visual system, supported by appropriate visualization tools, enables the human operator to solve complex tasks, like understanding and interpreting three-dimensional medical images, controlling air traffic by radar displays, supporting instrument flight tasks, or interacting with virtual realities. The development and application of new visualization techniques is of major importance for future technological progress.

Design recommendations for notifications are typically based on user performance and subjective feedback. In comparison, there has been surprisingly little research on how designed notifications might be processed by the brain for the information they convey. The current study uses EEG/ERP methods to evaluate auditory notifications that were designed to cue long-distance truck drivers for task-management and driving conditions, particularly for automated driving scenarios. Two experiments separately evaluated naive students and professional truck drivers for their behavioral and brain responses to auditory notifications, which were either auditory icons or verbal commands. Our EEG/ERP results suggest that verbal commands were more readily recognized by the brain as relevant targets, but that auditory icons were more likely to update contextual working memory. Both classes of notifications did not differ on behavioral measures. This suggests that auditory icons ought to be employed for communicating contextual information and verbal commands, for urgent requests.

This special interest group addresses the status quo of HCI research with regards to research practices of transparency and openness. Specifically, it discusses whether current practices are in line with the standards applied to other fields (e.g., psychology, economics, medicine). It seeks to identify current practices that are more progressive and worth communicating to other disciplines, while evaluating whether practices in other disciplines are likely to apply to HCI research constructively. Potential outcomes include: (1) a review of current HCI research policies, (2) a report on recommended practices, and (3) a replication project of key findings in HCI research. published

Take-over requests (TORs) in highly automated vehicles are cues that prompt users to resume control. TORs however, are often evaluated in non-moving driving simulators. This ignores the role of motion, an important source of information for users who have their eyes off the road while engaged in non-driving related tasks. We ran a user study in a moving-base driving simulator to investigate the effect of motion on TOR responses. We found that with motion, user responses to TORs vary depending on the road context where TORs are issued. While previous work showed that participants are fast to respond to urgent cues, we show that this is true only when TORs are presented on straight roads. Urgent cues issued on curved roads elicit slower responses than non-urgent cues on curved roads. Our findings indicate that TORs should be designed to be aware of road context to accommodate natural user responses.

Viewpoint selection has been an emerging area in computer graphics for some years, and it is now getting maturity with applications in fields such as scene navigation, scientific visualization, object recognition, mesh simplification, and camera placement. In this survey, we review and compare twenty-two measures to select good views of a polygonal 3D model, classify them using an extension of the categories defined by Secord et al., and evaluate them against the Dutagaci et al. benchmark. Eleven of these measures have not been reviewed in previous surveys. Three out of the five short-listed best viewpoint measures are directly related to information. We also present in which fields the different viewpoint measures have been applied. Finally, we provide a publicly available framework where all the viewpoint selection measures are implemented and can be compared against each other This work has been partially funded by grant TIN2016-75866-C3-3-R from the Spanish Government, grant 2017-SGR-1101 from Catalan Government and by the National Natural Science Foundation of China (Nos. 61571439, 61471261 and 61771335)

The cognitive workload of a steering task could reflect its demand on attentional as well as working memory resources under different conditions. These respective demands could be differentiated by evaluating components of the event-related potential (ERP) response to different types of stimulus probes, which are claimed to reflect the availability of either attention (i.e., novelty-P3) or working memory (i.e., target-P3) resources. Here, a within-subject analysis is employed to evaluate the robustness of ERP measurements in discriminating the cognitive demands of different steering conditions. We find that the amplitude of novelty-P3 ERPs to task-irrelevant environmental sounds is diminished when participants are required to perform a steering task. This indicates that steering places a demand on attentional resources. In addition, target-P3 ERPs to a secondary auditory detection task vary when the controller dynamics in the steering task are manipulated. This indicates that differences in controller dynamics vary in their working memory demands.

Early studies suggested that auditory stimuli are only (cognitively) processed if they are relevant for the task. However, studies that are more recent show that this is not necessarily true. Rather, it depends on the nature of the auditory stimuli. Environmental sounds are processed even when they are irrelevant for the task and even when participants are engaged in demanding visual tasks. These latter results can be interpreted within the framework of situational awareness. To maintain situational awareness, it is essential to continuously scan the environment for unexpected sounds that might not be of immediate task relevance but could inform us about important changes in the environment. In the current study, we investigate whether and how the scanning for, and processing of, environmental sounds—in other words, situational awareness of the auditory scene—is influenced by auditory attention manipulations. Here, auditory attention was manipulated by requiring participants to perform an auditory oddball-detection task or not, while task-irrelevant environmental sounds were occasionally presented in the background. The current study answers the following questions: (1) Is the processing of environmental sounds influenced by manipulations of auditory attention or is it an automatic process? (2) Is the processing of irrelevant environmental sounds attenuated or enhanced by auditory attention? On the one hand, the processing of the irrelevant environmental sounds could be attenuated, because the additional task increases the demand for auditory attentional resources. On the other hand, the processing of the environmental sounds could be enhanced because more attention is directed toward the auditory channel to optimize performance in the auditory oddball-detection task.

In this study, we demonstrate the effects of anxiety and cognitive load on eye movement planning in an instrument flight task adhering to a single-sensor-single-indicator data visualisation design philosophy. The task was performed in neutral and anxiety conditions, while a low or high cognitive load, auditory n-back task was also performed. Cognitive load led to a reduction in the number of transitions between instruments, and impaired task performance. Changes in self-reported anxiety between the neutral and anxiety conditions positively correlated with changes in the randomness of eye movements between instruments, but only when cognitive load was high. Taken together, the results suggest that both cognitive load and anxiety impact gaze behavior, and that these effects should be explored when designing data visualization displays.

It is increasingly viable to measure the brain activity of mobile users, as they go about their everyday business in their natural world environment. This is due to: (i) modern signal processing methods, (ii) lightweight and cost-effective measurement devices, and (iii) a better, albeit incomplete, understanding of how measurable brain activity relates to mental processes. Here, we address how brain activity can be measured in mobile users and how this contrasts with measurements obtained under controlled laboratory conditions. In particular, we will focus on electroencephalography (EEG) and will cover: (i) hardware and software implementation, (ii) signal processing techniques, (iii) interpretation of EEG measurements. This will consist of hands-on analyses of real EEG data and a basic theoretical introduction to how and why EEG works.