Search

Neuroergonomics as defined by Raja Parasuraman is the study of “the brain at work and in everyday life” (Parasuraman, 2003). This rapidly growing research field aims at understanding human brain function underlying the many facets of human interaction with technical systems (Dehais et al., 2020). The term “cognition” is used to describe different processes (e.g., attention, memory, decision making) relevant to human-technology interaction. Cognitive neuroergonomics, then, can be defined as a section of neuroergonomics concerned with the investigation of the neural bases of those cognitive processes involved in the user's interaction with a technical system at work or during everyday life. One of the defining aspects of cognitive neuroergonomics is that it uses insights from analyzing neural dynamics in these settings to inform cognitive theory and models, as well as to improve our understanding of human brain function underlying cognition, in general. To this end, new imaging methods are continuously adapted and used in a wide range of experimental scenarios that cover the entire area of ergonomics from highly controlled laboratory research protocols, to less controlled translational research, to research in the real world with little control over the factors of interest (Parada, 2018). This decreasing level of control is accompanied by an increasing level of ecological validity. Laboratory experiments provide very good control over experimental factors with high internal validity of the investigated constructs but often suffer from low levels of ecological validity. In contrast, real-world experiments might show low internal validity and lack of experimental control but provide high ecological validity that cannot be further improved. Here, the real world is the laboratory (Gramann et al., 2017). Furthermore, with increasing ecological validity, inter-acting with technical systems often involves expanding physical activity of the user. System interactions range from very low input (e.g., interaction with mobile devices; McKendrick, 2019) to larger scale interaction (e.g., Human-Robot Interaction, HRI; Tsarouchi et al., 2016) to very large scale interactions (e.g., assisted navigation; Wunderlich and Gramann, 2020). Active behavior is the basis for physically demanding workplaces as well as less physically challenging tasks that, nonetheless, require body, head and eye movements when users actively seek information or respond to external stimuli (Doshi and Trivedi, 2009). Traditionally, however, active behavior is not allowed in brain imaging protocols because established imaging modalities are usually too heavy to follow participants' movements and movement-related artifacts render the analyses of neural activity difficult (Makeig et al., 2009; Gramann et al., 2011). With cognitive neuroergonomics maturing into a new research area with widespread research questions and methods, the focus should be put back into theory-driven studies of the human brain at work and in everyday life. Good scientific practices have to be adapted to allow for replicable science including the integration of new mobile imaging methods into the existing range of established imaging protocols. New findings have to be related to parameters known from established laboratory protocols and integrated into larger theoretical frameworks that allow for systematic replication as well as the development of robust parameters reflecting cognitive processes. From this perspective, it is our belief that the following challenges will have to be met to further develop this scientific field.

The NASA task load index (TLX) is the most used tool for measuring mental workload. The tools widespread use is associated with its pedigree, and its simplicity of application and interpretation. However, the NASA TLX has been criticized for not measuring the mental construct it purports to measure. The particular claim is that the NASA TLX does not measure perceived mental workload, instead it measures perceived task difficulty. We explored this claim using a simple spatial memory task. The results were analyzed with polynomial hierarchical regression. The NASA TLX failed the test of construct validity and, the sub-components each had random interpretations depending on the participant. The second point is particularly troubling, as many system designs could be based on random, non-causal relationships. Both these observations raise doubts in toward the use of the NASA TLX as a calibrated instrument for human factors engineering.

Objective: The aim of this study was to assess performance and cognitive states during cognitive work in the presence of physical work and in natural settings. Background: Authors of previous studies have examined the interaction between cognitive and physical work, finding performance decrements in working memory. Neuroimaging has revealed increases and decreases in prefrontal oxygenated hemoglobin during the interaction of cognitive and physical work. The effect of environment on cognitive-physical dual tasking has not been previously considered. Method: Thirteen participants were monitored with wireless functional near-infrared spectroscopy (fNIRS) as they performed an auditory 1-back task while sitting, walking indoors, and walking outdoors. Results: Relative to sitting and walking indoors, auditory working memory performance declined when participants were walking outdoors. Sitting during the auditory 1-back task increased oxygenated hemoglobin and decreased deoxygenated hemoglobin in bilateral prefrontal cortex. Walking reduced the total hemoglobin available to bilateral prefrontal cortex. An increase in environmental complexity reduced oxygenated hemoglobin and increased deoxygenated hemoglobin in bilateral prefrontal cortex. Conclusion: Wireless fNIRS is capable of monitoring cognitive states in naturalistic environments. Selective attention and physical work compete with executive processing. During executive processing loading of selective attention and physical work results in deactivation of bilateral prefrontal cortex and degraded working memory performance, indicating that physical work and concomitant selective attention may supersede executive processing in the distribution of mental resources. Application: This research informs decision-making procedures in work where working memory, physical activity, and attention interact. Where working memory is paramount, precautions should be taken to eliminate competition from physical work and selective attention.

There are a number of key data-centric questions that must be answered when developing classifiers for operator functional states. “Should a supervised or unsupervised learning approach be used? What degree of labeling and transformation must be performed on the data? What are the trade-offs between algorithm flexibility and model interpretability, as generally these features are at odds?” Here, we focus exclusively on the labeling of cognitive load data for supervised learning. We explored three methods of labeling cognitive states for three-state classification. The first method labels states derived from a tertiary split of trial difficulty during a spatial memory task. The second method was more adaptive; it employed a mixed-effects stress–strain curve and estimated an individual’s performance asymptotes with respect to the same spatial memory task. The final method was similar to the second approach; however, it employed a mixed-effects Rasch model to estimate individual capacity limits within the context of item response theory for the spatial memory task. To assess the strength of each of these labeling approaches, we compared the area under the curve (AUC) for receiver operating curves (ROCs) from elastic net and random forest classifiers. We chose these classifiers based on a combination of interpretability, flexibility, and past modeling success. We tested these techniques across two groups of individuals and two tasks to test the effects of different labeling techniques on cross-person and cross-task transfer. Overall, we observed that the Rasch model labeling paired with a random forest classifier led to the best model fits and showed evidence of both cross-person and cross-task transfer.

As semiautonomous driving systems are becoming prevalent in late model vehicles, it is important to understand how such systems affect driver attention. This study investigated whether measures from low-cost devices monitoring peripheral physiological state were comparable to standard EEG in predicting lapses in attention to system failures. Twenty-five participants were equipped with a low-fidelity eye-tracker and heart rate monitor and with a high-fidelity NuAmps 32-channel quick-gel EEG system and asked to detect the presence of potential system failure while engaged in a fully autonomous lane changing driving task. To encourage participant attention to the road and to assess engagement in the lane changing task, participants were required to: (a) answer questions about that task; and (b) keep a running count of the type and number of billboards presented throughout the driving task. Linear mixed effects analyses were conducted to model the latency of responses reaction time (RT) to automation signals using the physiological metrics and time period. Alpha-band activity at the midline parietal region in conjunction with heart rate variability (HRV) was important in modeling RT over time. Results suggest that current low-fidelity technologies are not sensitive enough by themselves to reliably model RT to critical signals. However, that HRV interacted with EEG to significantly model RT points to the importance of further developing heart rate metrics for use in environments where it is not practical to use EEG.

Studying the brain in complex work settings is beneficial to how we develop advanced technology and how we integrate mobility into cognitive work. This chapter reviews two studies examining how the brain responds to environmental changes and how it reacts to mobile use of computing devices. Initial findings are expanded upon by relating effects of selective attention and performance across both studies while also exploring issues of pedestrian safety, creativity, and efficiency at work.

The present experiments examined the effects of two neuroergonomic methods—molecular genetics and transcranial Direct Current Stimulation (tDCS)—on skill acquisition in complex multitasking environments. The results of the first experiment showed that a variant of a common dopamine gene, COMT, interacted with training in supervisory control of unmanned vehicles. Individuals with the Met allele of the COMT gene showed greater trained-related gains in performance than those with the Val allele, consistent with the role of COMT in regulating dopamine availability and executive function. In the second experiment, stimulation of dorsal and ventral frontoparietal attention networks with tDCS accelerated acquisition of multitasking skill on the Space Fortress game. These findings indicate that novel neuroergonomic methods can supplement more traditional training methods for the development of expertise in complex tasks.

Functional near infrared spectroscopy (fNIRS) is an emerging neuroimaging technique that has found home in various human factors and ergonomics applications. Why fNIRS? Is it better than EEG or fMRI? Is it an appropriate neuroimaging technique for my research/application? What are the methodological considerations for fNIRS analyses? This panel discussion is aimed at answering these questions, among others, when panelists from varied human factors and ergonomics applications discuss how they employ fNIRS in their investigations.

The chapter aims at analyzing the impact of physical work on brain activity and performance while engaged in cognitive work. Working-memory performance has been found to be lessened with concurrent physical activity. Prefrontal oxygenated hemoglobin varies widely according to fNIRS studies. The influence of environment has been neglected, however. To learn how the environment affects physical work concurrent with mental activity, participants were asked to combine auditory tasks with walking indoors and out. for which results were recorded.

The chapter aims at analyzing the impact of highly mobile computing devices on the situational awareness and mental workload of users navigating a college campus. Such devices could have a positive impact on quality of life, productivity, and performance. However, these concepts are difficult to measure. For this purpose, hemodynamic measures were combined with handheld displays and augmented-reality wearable displays, along with vision and hearing, to provide a behavioral assessment.

The chapter aims at analyzing the impact of workload transitions on behavior and prefrontal brain activity to test the hypothesis that workload transitions are characterized by cognitive disruption and dissatisfaction. For this purpose, this hypothesis was tested and results were recorded.

The chapter aims at analyzing the impact of noninvasive brain stimulation via transcranial direct current stimulation (tDCS) on brain activity while individuals perform a spatial working-memory task. Both an experimental group and a control group were subjected to multiple sessions of sham and actual stimulation as observed via functional near-infrared spectroscopy (fNIRS) of the prefrontal cortex and a finite-element analysis of current flow to this brain region, and results were recorded.

A spatial memory task with an event related design was used as a proxy for showcasing a design matrix regression on fNIRS data for neuroergonomic studies of complex tasks involving transient workload transitions. The analysis identified a region of ventrolateral prefrontal cortex (VLPFC) in which oxygenated hemoglobin (HbO2) was increased during memory maintenance. Task performance negatively modulated HbO2 in this brain region during memory maintenance. The results show that neural effects specifically within the period of memory maintenance, as opposed to other cognitive components (e.g., encoding or retrieval) can be reliably identified. Similar analytical methods could be used for the analysis of cortical hemodynamics in more complex tasks involving transient events (e.g., workload transitions) and in related neuroergonomic applications.

Enhancing working memory through brain stimulation can contribute to the neuroergonomic goal of improving cognitive functioning at work and in everyday life. Imaging and lesion studies suggest that verbal and spatial working memory processing may be controlled in separate brain regions. Specifically, verbal working memory has been associated with the left dorsolateral prefrontal cortex (DLPFC) and spatial working memory with the right DLPFC. The current study used noninvasive brain stimulation to further examine this cortical dissociation between verbal and spatial working memory. We administered 2mA or 0.1mA of anodal transcranial direct current stimulation (tDCS) to the left and right DLPFC while participants performed verbal and spatial working memory tasks. Contrary to our initial hypotheses, stimulation of the left DLPFC improved spatial working memory but not verbal working memory. Furthermore, stimulation of right DLPFC did not affect verbal or spatial working memory.

Objective: Assess team performance within a networked supervisory control setting while manipulating automated decision aids and monitoring team communication and working memory ability. Background: Networked systems such as multi–unmanned air vehicle (UAV) supervision have complex properties that make prediction of human-system performance difficult. Automated decision aid can provide valuable information to operators, individual abilities can limit or facilitate team performance, and team communication patterns can alter how effectively individuals work together. We hypothesized that reliable automation, higher working memory capacity, and increased communication rates of task-relevant information would offset performance decrements attributed to high task load. Method: Two-person teams performed a simulated air defense task with two levels of task load and three levels of automated aid reliability. Teams communicated and received decision aid messages via chat window text messages. Results: Task Load × Automation effects were significant across all performance measures. Reliable automation limited the decline in team performance with increasing task load. Average team spatial working memory was a stronger predictor than other measures of team working memory. Frequency of team rapport and enemy location communications positively related to team performance, and word count was negatively related to team performance. Conclusion: Reliable decision aiding mitigated team performance decline during increased task load during multi-UAV supervisory control. Team spatial working memory, communication of spatial information, and team rapport predicted team success. Application: An automated decision aid can improve team performance under high task load. Assessment of spatial working memory and the communication of task-relevant information can help in operator and team selection in supervisory control systems.

We interact daily with computers that appear and behave like humans. Some researchers propose that people apply the same social norms to computers as they do to humans, suggesting that social psychological knowledge can be applied to our interactions with computers. In contrast, theories of human–automation interaction postulate that humans respond to machines in unique and specific ways. We believe that anthropomorphism—the degree to which an agent exhibits human characteristics—is the critical variable that may resolve this apparent contradiction across the formation, violation, and repair stages of trust. Three experiments were designed to examine these opposing viewpoints by varying the appearance and behavior of automated agents. Participants received advice that deteriorated gradually in reliability from a computer, avatar, or human agent. Our results showed (a) that anthropomorphic agents were associated with greater trust resilience, a higher resistance to breakdowns in trust; (b) that these effects were magnified by greater uncertainty; and c) that incorporating human-like trust repair behavior largely erased differences between the agents. Automation anthropomorphism is therefore a critical variable that should be carefully incorporated into any general theory of human–agent trust as well as novel automation design.

Previous research has shown that there is a time cost (i.e., a resumption lag) associated with resuming a task following an interruption and that the longer the duration of the interruption, the greater the time cost (i.e., resumption lag increases as interruption duration increases). The memory-for-goals model (Altmann & Trafton, 2002) suggests that this greater time cost is a result of increased interference caused by longer duration interruptions. Therefore, the goal for this research was to determine whether individuals who can better manage interference, i.e., individuals with higher working-memory capacity (WMC), can resume tasks more quickly following interruptions than those who cannot manage interference as well (i.e., individuals with lower WMC). A procedural interruption task with 3 different interruption durations and a measure of WMC were completed by 229 students. In line with previous research, we found a strong positive relationship between interruption duration and resumption lag. We found a strong negative effect of WMC on resumption lag (i.e., increases in WMC reduced resumption lags). Notably, WMC moderated the effect of interruption duration on resumption lag (i.e., increases in WMC attenuated the positive relationship between interruption duration and resumption lag). Specifically, individuals with high WMC experienced small increases in resumption lag as interruption duration increased, whereas individuals with low WMC experienced substantial increases in resumption lag as interruption duration increased. Our data suggest that individuals with higher WMC are less susceptible to interference caused by interruptions. (PsycINFO Database Record

The effects of two variable priority (or emphasis change) training paradigms (with 3 or 5 levels of emphasis) and an adaptive training paradigm on skill acquisition in dual verbal-spatial working memory tasks were examined. Competing hypotheses of the cognitive mechanisms of variable priority training were evaluated. All three training groups showed improved performance over a control group on the verbal working memory task. Only the variable priority groups showed an improvement over time relative to the control group in the spatial working memory task. The results support the hypothesis that variable priority training forces trainees to focus not only on easy tasks, but also on harder tasks, yielding more balanced task performance and higher levels of skill development. No evidence was found that variable priority training leads to the discovery of an optimal attention allocation strategy, or more efficient adaptation to changing task demands.

The effects of task load, automation reliability and team communication on supervisory control performance were examined using a multi-UAV simulation with two operators. Performance was degraded by...

Highly mobile computing devices promise to improve quality of life, productivity, and performance. Increased situation awareness and reduced mental workload are two potential means by which this can be accomplished. However, it is difficult to measure these concepts in the “wild”. We employed ultra-portable battery operated and wireless functional near infrared spectroscopy (fNIRS) to non-invasively measure hemodynamic changes in the brain’s Prefrontal cortex (PFC). Measurements were taken during navigation of a college campus with either a hand-held display, or an Augmented reality wearable display (ARWD). Hemodynamic measures were also paired with secondary tasks of visual perception and auditory working memory to provide behavioral assessment of situation awareness and mental workload. Navigating with an augmented reality wearable display produced the least workload during the auditory working memory task, and a trend for improved situation awareness in our measures of prefrontal hemodynamics. The hemodynamics associated with errors were also different between the two devices. Errors with an augmented reality wearable display were associated with increased prefrontal activity and the opposite was observed for the hand-held display. This suggests that the cognitive mechanisms underlying errors between the two devices differ. These findings show fNIRS is a valuable tool for assessing new technology in ecologically valid settings and that ARWDs offer benefits with regards to mental workload while navigating, and potentially superior situation awareness with improved display design.

Contemporary studies with transcranial direct current stimulation (tDCS) provide a growing base of evidence for enhancing cognition through the non-invasive delivery of weak electric currents to the brain. The main effect of tDCS is to modulate cortical excitability depending on the polarity of the applied current. However, the underlying mechanism of neuromodulation is not well understood. A new generation of functional near infrared spectroscopy (fNIRS) systems is described that are miniaturized, portable, and include wearable sensors. These developments provide an opportunity to couple fNIRS with tDCS, consistent with a neuroergonomics approach for joint neuroimaging and neurostimulation investigations of cognition in complex tasks and in naturalistic conditions. The effects of tDCS on complex task performance and the use of fNIRS for monitoring cognitive workload during task performance are described. Also explained is how fNIRS + tDCS can be used simultaneously for assessing spatial working memory. Mobile optical brain imaging is a promising neuroimaging tool that has the potential to complement tDCS for realistic applications in natural settings.

There is a need to facilitate acquisition of real world cognitive multi-tasks that require long periods of training (e.g., air traffic control, intelligence analysis, medicine). Non-invasive brain stimulation—specifically transcranial Direct Current Stimulation (tDCS)—has promise as a method to speed multi-task training. We hypothesized that during acquisition of the complex multi-task Space Fortress, subtasks that require focused attention on ship control would benefit from tDCS aimed at the dorsal attention network while subtasks that require redirection of attention would benefit from tDCS aimed at the right hemisphere ventral attention network. We compared effects of 30 min prefrontal and parietal stimulation to right and left hemispheres on subtask performance during the first 45 min of training. The strongest effects both overall and for ship flying (control and velocity subtasks) were seen with a right parietal (C4, reference to left shoulder) montage, shown by modeling to induce an electric field that includes nodes in both dorsal and ventral attention networks. This is consistent with the re-orienting hypothesis that the ventral attention network is activated along with the dorsal attention network if a new, task-relevant event occurs while visuospatial attention is focused (Corbetta et al., 2008). No effects were seen with anodes over sites that stimulated only dorsal (C3) or only ventral (F10) attention networks. The speed subtask (update memory for symbols) benefited from an F9 anode over left prefrontal cortex. These results argue for development of tDCS as a training aid in real world settings where multi-tasking is critical.

To better understand the mechanisms by which working memory training can augment human performance we continuously monitored trainees with near infrared spectroscopy (NIRS) while they performed a dual verbal-spatial working memory task. Linear mixed effects models were used to model the changes in cerebral hemodynamic response as a result of time spent training working memory. Nonlinear increases in left dorsolateral prefrontal cortex (DLPFC) and right ventrolateral prefrontal cortex (VLPFC) were observed with increased exposure to working memory training. Adaptive and yoked training groups also showed differential effects in rostral prefrontal cortex with increased exposure to working memory training. There was also a significant negative relationship between verbal working memory performance and bilateral VLPFC activation. These results are interpreted in terms of decreased proactive interference, increased neural efficiency, reduced mental workload for stimulus processing, and increased working memory capacity with training.

The visual system is an efficient statistician, extracting statistical summaries over sets of objects (statistical summary perception) and statistical regularities among individual objects (statistical learning). Although these two kinds of statistical processing have been studied extensively in isolation, their relationship is not yet understood. We first examined how statistical summary perception influences statistical learning by manipulating the task that participants performed over sets of objects containing statistical regularities (Experiment 1). Participants who performed a summary task showed no statistical learning of the regularities, whereas those who performed control tasks showed robust learning. We then examined how statistical learning influences statistical summary perception by manipulating whether the sets being summarized contained regularities (Experiment 2) and whether such regularities had already been learned (Experiment 3). The accuracy of summary judgments improved when regularities were removed and when learning had occurred in advance. In sum, calculating summary statistics impeded statistical learning, and extracting statistical regularities impeded statistical summary perception. This mutual interference suggests that statistical summary perception and statistical learning are fundamentally related.