Search

As semiautonomous driving systems are becoming prevalent in late model vehicles, it is important to understand how such systems affect driver attention. This study investigated whether measures from low-cost devices monitoring peripheral physiological state were comparable to standard EEG in predicting lapses in attention to system failures. Twenty-five participants were equipped with a low-fidelity eye-tracker and heart rate monitor and with a high-fidelity NuAmps 32-channel quick-gel EEG system and asked to detect the presence of potential system failure while engaged in a fully autonomous lane changing driving task. To encourage participant attention to the road and to assess engagement in the lane changing task, participants were required to: (a) answer questions about that task; and (b) keep a running count of the type and number of billboards presented throughout the driving task. Linear mixed effects analyses were conducted to model the latency of responses reaction time (RT) to automation signals using the physiological metrics and time period. Alpha-band activity at the midline parietal region in conjunction with heart rate variability (HRV) was important in modeling RT over time. Results suggest that current low-fidelity technologies are not sensitive enough by themselves to reliably model RT to critical signals. However, that HRV interacted with EEG to significantly model RT points to the importance of further developing heart rate metrics for use in environments where it is not practical to use EEG.

Many everyday tasks, such as driving and reading technical reports, require sustained attention. These tasks may deplete attentional resources and/or lead to mind wandering, boredom and fatigue – cognitive states that result in impaired performance. Numerous competing theories exist to explain these performance decrements. In this panel, we will examine perspectives from both the fatigue and vigilance literatures, identifying points of common ground and differences in an attempt to further understand the underpinnings of performance decrements in tasks that require sustained attention. Experts in the interrelated yet frequently viewed as disparate areas of fatigue and vigilance are brought together to forge new theoretical ground.

In information dense, visually complex environments (e.g., automobiles, airplanes, and operating rooms) auditory alarms can direct attention to critical events. Effective alarms are designed to convey a hazard level appropriate for the situation they represent. Acoustic intensity plays an essential role in perceived urgency, with louder sounds generally being perceived as representing something more urgent. However, intensity may be dictated by factors outside the alarm designer’s control (e.g., background noise, manufacturer’s sound system specifications). Therefore it is essential to examine other acoustic parameters that can be used to convey scalable levels of perceived urgency. Previous work suggests that looming sounds, or sounds perceived to be coming towards a listener, may result in faster response times than other types of sounds. Further, people may respond faster to looming sounds if the rate of change (ROC) in the increase in intensity is more rapid, potentially making them a strong candidate for scalability. Sounds increasing in intensity quickly may be perceived as more urgent than other sounds. However, it is critical that ROC is not confounded by overall intensity, since the intensity-urgency relationship is well documented. The present study measured perceived urgency ratings of auditory looming stimuli with different intensity ROCs that were equated for overall intensity. Results revealed no differences in urgency ratings across the three ROCs. Current results indicate that overall intensity is more critical than ROC to achieving scalability in auditory alarms.

Mind wandering is a pervasive threat to transportation safety, potentially accounting for a substantial number of crashes and fatalities. In the current study, mind wandering was induced through completion of the same task for 5 days, consisting of a 20-minute monotonous freeway-driving scenario, a cognitive depletion task, and a repetition of the 20-minute driving scenario driven in the reverse direction. Participants were periodically probed with auditory tones to self-report whether they were mind wandering or focused on the driving task. Self-reported mind wandering frequency was high, and did not statistically change over days of participation. For measures of driving performance, participant labeled periods of mind wandering were associated with reduced speed and reduced lane variability, in comparison to periods of on task performance. For measures of electrophysiology, periods of mind wandering were associated with increased power in the alpha band of the electroencephalogram (EEG), as well as a reduction in the magnitude of the P3a component of the event related potential (ERP) in response to the auditory probe. Results support that mind wandering has an impact on driving performance and the associated change in driver’s attentional state is detectable in underlying brain physiology. Further, results suggest that detecting the internal cognitive state of humans is possible in a continuous task such as automobile driving. Identifying periods of likely mind wandering could serve as a useful research tool for assessment of driver attention, and could potentially lead to future in-vehicle safety countermeasures.

Tactile displays hold promise as an effective and efficient means of presenting a wide range of information to the driver. This study examined the subjective perceptions of urgency and annoyance for tactile signals of different pulse duration, interpulse interval (IPI), and pulse pattern from devices called tactors located on the wrist, on the waist belt, or in the seat pan. Results revealed significant utility (steep increases in urgency without similarly steep increases in annoyance) for signals presented in pulse durations from 10 to 150 ms with decreasing utility beyond this range (200 ms or greater). Perceived urgency showed a decreasing trend as the IPI increased. Strikingly similar magnitude estimation functions were obtained across the three tactor locations. Results are discussed in terms of their implications for tactile display design in vehicles.

Participants drove a simulated vehicle while completing an n-back task of high or low cognitive demand. Concurrently, they responded via brake press to signals presented in either a single modality (auditory, visual or tactile) or a bimodal combination. Participants were asked to indicate the perceived urgency of each signal by adjusting the force applied in their brake response. Signals were designed to be of “high” or “low” urgency based on previous research for both unimodal and bimodal combinations. Participants were capable of subjectively judging perceived urgency independent of response time and responded to bimodal (relative to unimodal) and high (relative to low) urgency stimuli with significantly greater brake force. However, both factors interacted with cognitive demand indicating that the magnitude of these differential responses was reduced under high demand. Implications for warning evaluation in an in-vehicle driving context are discussed.

The use of machine learning algorithms to classify Mental Workload (MW) from various neurophysiological measures is a growing trend in Human Factors research. Several teams have demonstrated that artificial neural networks (ANNs) can be employed to differentiate highly-dimensional neurophysiological data from participants performing tasks at different levels of cognitive demand in many experimental paradigms. Yet, in other cases, classifier performance was found not to exceed chance levels. One inescapable aspect of neurophysiological measures is the time course associated with data collection. This study directly examined the effect of time on classifier performance. Relative classifier performance values were compared for ANNs trained with EEG data from participants performing a verbal/spatial n-back task at varying load levels. For the vast majority of participants, classifiers trained with data from one session were ineffective when tested with data from a subsequent session. Similarly, classification performance suffered when training and testing tasks were incongruent. In contrast to the very high aggregate performance when trained and tested with data from the same session and same task (M = .81), cross-classification performance was consistently much worse (M = .49 – .53). The relative magnitude of the cross-session and cross-task “costs” to classification were compared, revealing a greater effect for session than task. The authors argue that the mere passage of time causes tonic changes in MW-related features of EEG that severely confound ANNs.

Tactile vibrations are potentially useful in a variety of environments to communicate information to visually and auditorily overloaded people. However, since vibrotactile signals must come into physical contact with the skin, they may also be perceived as highly urgent and annoying. The current study examined whether scalable levels of perceived urgency could be obtained with tactile signals by measuring the relationship between changes in vibrotactile pulse rate and ratings of urgency and annoyance. In two separate experiments, changes in pulse rate resulted in changes in ratings of perceived urgency with faster pulse rates being perceived as more urgent. Importantly, in both studies pulse rate had a greater impact on perceived urgency than it did on annoyance suggesting that scalable levels of urgency can be achieved without similarly annoying operators. Results are discussed in terms of their implications for multimodal display design.

Through a series of investigations involving different levels of contextual fidelity we devel- oped scales of perceived urgency for several dimensions of the auditory, visual, and tactile modalities. Psychophysical ratings of perceived urgency, annoyance, and acceptability as well as behavioral re- sponses to signals in each modality were obtained and analyzed using Steven's Power Law to allow comparison across modalities. Obtained results and their implications for use as in-vehicle alerts and warnings are discussed.

Age-related difficulties in speech processing remain a concern, especially as technology continues to depend heavily on successful speech comprehension on the part of users. Event-related potentials (ERPs) have frequently been used to assess age-related changes in the processing of language. Specifically, the amplitude of the ERP is often compared between conditions or groups of interest. In constructing ERPs, many neurophysiologic responses are averaged together to reduce the contribution of uncorrelated background activity in the electroencephalogram (EEG). However, if variability in the timing of each potential on a trial-by-trial basis (i.e., “latency jitter”) is confounded with a variable of interest, then the presence of amplitude differences observed in the average ERP might not be solely the result of genuine amplitude differences, but also timing variability. We examined the role latency jitter may play in the well-established observation of age-related changes in the processing of natural speech...

Electroencephalography (EEG) has the prospect of providing a means to gauge operator workload in a manner that does not intrude on the task being performed. Specifically, it has been proposed that the technique could be used as a method to speed the learning of a task, by adjusting the task to suit the state of the learner. The present study recorded EEG while participants performed a simulated Unmanned Aerial Vehicle (UAV) reconnaissance task. Analysis of power in three EEG frequency bands of interest found differences between the types of task being performed; however more complex analysis may be necessary to discern levels of difficulty within the task.

This experiment compared simulator sickness between males and females as a function of fixed-base versus rotating base platforms. Eight males and eight females drove through two routes in a driving simulator. One route was presented in fixed-base mode and another was presented in motion-base with a .5 to 1 ratio of motion (physical world to virtual world). Routes and fixed versus motion mode were presented in counter- balanced order. Measures of simulator sickness on the Simulator Sickness Questionnaire (Kennedy et al., 1993) were obtained after each route. As predicted, males reported lower levels of simulator sickness than females. A nonsignificant trend for this gender effect to be diminished in the rotating versus stationary condition was observed. Results warrant additional investigation into the potential for motion-based platforms to reduce the incidence and severity of simulator sickness in populations at greater risk of experiencing these negative consequences (i.e., females and older adults).

Twenty-one young (M=21.3) and 19 older (M=73.1) adults completed an aurally administered test of working memory capacity presented at 5 above threshold levels (65, 60, 55, 50, and 45 dB). The test required verification of sentences and recall of sentence final words at the end of each set. Audiometric assessment as well as sentence verification ensured that all auditory stimuli were above hearing levels in both age groups. Both young and older listeners demonstrated reduced working memory capacity scores as stimuli were presented at lower dB levels. The impact of decreases in dB level was greater for older relative to younger adults. But since older adults have elevated hearing thresholds within the range investigated here, the capacity differences observed between the two groups may be negligible. Implications of the results for theories of cognitive aging and for cognitive testing among older adults with subclinical hearing loss are discussed.

Driver fatigue is associated with increased lane position variability, decreased response time and increased crash rate. Collision avoidance systems (CASs), which are currently installed in many vehicles, have been shown to reduce the crash rates of alert drivers. The current investigation examined the effectiveness of two auditory CAS warnings for reducing crash rates in fatigued young and older drivers faced with a potential rear-end collision. Relative to a no-warning condition, auditory CAS warnings reduced overall crash probability. The presence of a CAS warning was particularly beneficial to reducing crashes in older drivers. Results of this study indicate that auditory CAS warnings may help reduce fatigue-related rearend crashes, particularly among older drivers. Fatigue and sleepiness account for

Advanced technologies have enabled the choice of either visual or auditory formats for avionics and surface transportation displays. Methods of assessing the mental workload imposed by displays of different formats are critical to their successful implementation. Towards this end a series of investigations were conducted with the following aims: 1) developing analogous auditory and visual versions of a secondary task that could be used to compare display modalities; and 2) to compare the sensitivity of neurophysiological, behavioral and subjective indices of workload. Experiments 1 and 2 confirmed that analogous auditory and visual secondary oddball discrimination tasks were of equivalent difficulty as indicated by P300 amplitude, RT, accuracy and subjective ratings of workload. Experiments 1–3 revealed that RT and accuracy for target detections were generally more sensitive to changes in primary task difficulty than P300 responses and subjective ratings. However, Experiment 3 indicated that P300 amplitude was sensitive to increased perceptual demands (resulting from driving in heavy fog versus clear visibility) not revealed by changes in either behavioral or subjective indices. Together the results of the current investigations indicate that a battery of assessment techniques will provide the most sensitive assessment of workload in complex environments.

Despite extensive examination of display parameters affecting written and nonverbal warnings, few investigators have examined presentation parameters affecting auditorially presented verbal warnings. This investigation is the first systematic examination of display parameters affecting verbal warnings that required listeners to be engaged in a contextually appropriate simultaneous task. Participants responded to verbal collision avoidance system (CAS) messages while engaged in a simultaneous simulated driving task. Perceived urgency, alerting effectiveness and annoyance ratings and behavioral reaction times were examined as a function of the signal word, CAS message, and presentation level (PL). Consistent with predictions from previous research involving written warnings, differences in the perceived urgency were observed as a function of the signal word used. PL also significantly impacted perceived urgency and alerting effectiveness. CAS messages presented at a S/N ratio of at least +4 dB resulted in high ratings of perceived urgency ratings and were responded too relatively quickly without a substantial tradeoff in increased annoyance ratings.

Drivers over the age of 65 are increasing rapidly in numbers and are driving more frequently. It is well recognized that older driver safety may be compromised due to age-related changes in cognitive functioning. Less well appreciated, however, is that sensory loss may interact with cognitive changes to compound the threat to safety. A sensory-cognitive interaction theory of ageing is described and its implications are discussed for the design of in-vehicle technologies. A particular area of focus is the difficulties of older drivers in carrying out navigational tasks while simultaneously maintaining safe control of the motor vehicle. Advanced automotive technologies including in-vehicle route guidance and navigational displays have the potential to improve safety and mobility if designed in accordance with the sensory/cognitive abilities of older drivers.

Recent empirical evidence underscoring the necessity of considering the role hearing abilities when using auditory tasks to assess age-related changes in cognitive abilities is presented. Young participants, screened for hearing impairment, were required to perform a simulated driving task concurrently with an auditory task in which stimuli were presented at intensity levels simulating age-related hearing impairment. Young participants demonstrated performance decrements mirroring those of older participants in a separate but similar dual task investigation. In conjunction with evidence found by previous researchers, the results presented here provide clear evidence for the need to utilize methodologies that account for age-related hearing deficit when using auditory tasks as indices of cognitive abilities with older adults.

Recent work has identified an event-related potential (ERP) component, the incongruency negativity (Ninc), which is sensitive to auditory Stroop conflict processing. Here, we investigated how this index of conflict processing is influenced by individual differences in cognitive style. There is evidence that individuals differ in the strategy they use to navigate through the environment; some use a predominantly verbal-egocentric strategy while others rely more heavily on a spatial-allocentric strategy. In addition, navigational strategy, assessed by a way-finding questionnaire, is predictive of performance on an auditory spatial Stroop task, in which either the semantic or spatial dimension of stimuli must be ignored. To explore the influence of individual differences in navigational style on conflict processing, participants took part in an auditory spatial Stroop task while the electroencephalogram (EEG) was recorded. Whereas behavioral performance only showed a main effect of congruency, we observed the predicted three-way interaction between congruency, task type and navigational style with respect to our physiological measure of Stroop conflict. Specifically, congruency-dependent modulation of the Ninc was observed only when participants performed their non-dominant task (e.g., verbal navigators attempting to ignore semantic information). These results confirm that the Ninc reliably indexes auditory Stroop conflict and extend previous results by demonstrating that the Ninc is predictably modulated by individual differences in cognitive style.

Hazard mapping is essential to effective driver-vehicle interface (DVI) design. Determining which modality to use for situations of different criticality requires an understanding of the relative impact of signal parameters within each modality on perceptions of urgency and annoyance. Towards this goal we obtained psychometric functions for visual, auditory and tactile interpulse interval (IPI), visual color, signal word, and auditory fundamental frequency on perceptions of urgency, annoyance, and acceptability. Results indicate that manipulation of IPI in the tactile modality, relative to visual and auditory, has greater utility (greater impact on urgency than annoyance). Manipulations of color were generally rated as less annoying and more acceptable than auditory and tactile stimuli; but they were also rated as lower in urgency relative to other modality manipulations. Manipulation of auditory fundamental frequency resulted in high ratings of both urgency and annoyance. Results of the current investigation can be used to guide DVI design and evaluation.

Matching the perceived urgency of an alert with the relative hazard level of the situation is critical for effective alarm response. Two experiments describe the impact of acoustic and semantic parameters on ratings of perceived urgency, annoyance and alerting effectiveness and on alarm response speed. Within a simulated driving context, participants rated and responded to collision avoidance system (CAS) messages spoken by a female or male voice (experiments 1 and 2, respectively). Results indicated greater perceived urgency and faster alarm response times as intensity increased from -2 dB signal to noise (S/N) ratio to +10 dB S/N, although annoyance ratings increased as well. CAS semantic content interacted with alarm intensity, indicating that at lower intensity levels participants paid more attention to the semantic content. Results indicate that both acoustic and semantic parameters independently and interactively impact CAS alert perceptions in divided attention conditions and this work can inform auditory alarm design for effective hazard matching. Matching the perceived urgency of an alert with the relative hazard level of the situation is critical for effective alarm response. Here, both acoustic and semantic parameters independently and interactively impacted CAS alert perceptions in divided attention conditions. This work can inform auditory alarm design for effective hazard matching. STATEMENT OF RELEVANCE: Results indicate that both acoustic parameters and semantic content can be used to design collision warnings with a range of urgency levels. Further, these results indicate that verbal warnings tailored to a specific hazard situation may improve hazard-matching capabilities without substantial trade-offs in perceived annoyance.

Adaptive training using neurophysiological measures requires efficient classification of mental workload in real time as a learner encounters new and increasingly difficult levels of tasks. Previous investigations have shown that artificial neural networks (ANNs) can accurately classify workload, but only when trained on neurophysiological exemplars from experienced operators on specific tasks. The present study examined classification accuracies for ANNs trained on electroencephalographic (EEG) activity recorded while participants performed the same (within task) and different (cross) tasks for short periods of time with little or no prior exposure to the tasks. Participants performed three working memory tasks at two difficulty levels with order of task and difficulty level counterbalanced. Within-task classification accuracies were high when ANNs were trained on exemplars from the same task or a set containing the to-be-classified task, (M = 87.1% and 85.3%, respectively). Cross-task classification accuracies were significantly lower (average 44.8%) indicating consistent systematic misclassification for certain tasks in some individuals. Results are discussed in terms of their implications for developing neurophysiologically driven adaptive training platforms.

Technological advances allow presentation of more information (e.g. alerts, warnings) than operators can process. Effective management of this information requires designing displays and warnings that scale the perceived urgency of the signal to the hazard level it represents. We reviewed existing methods of scaling and propose that determining a user-defined range of acceptable intensities prior to implementing the scaling (the CRASINS method) is most effective. Here, we used the CRASINS method to develop urgency and annoyance scales for visual, auditory, and tactile signals. Parameters examined included interpulse interval (IPI) and pulse duration in the auditory and tactile modalities as well as frequency for auditory and colour for visual stimuli. The greatest range of urgency was obtained with the auditory modality, while visual stimuli result in the lowest levels of annoyance. Manipulations of IPI yielded good utility in all modalities with the highest utility being observed for changes in colour.

Previous empirical research has demonstrated that even within clearly audible ranges (45-70 dB with less than 35 dB background noise), people tend to make more errors and take longer to respond to even simple auditory tasks as the presentation level of auditory stimuli decreases when attentional processing requirements are high (Baldwin, 1997, Baldwin, Struckman-Johnson, Galinsky, & Williams, 1999). The observed decrease in performance with decreasing decibel level may be due to shorter echoic memory traces, which could exacerbate demands on attentional resources during periods of high mental workload. In order to examine this hypothesis, participants were asked to listen to sets of tones and verbally indicate whether the second pattern in each set matched the first pattern while simultaneously performing a loading task involving simulated driving. Tonal sets varied according to the decibel level of presentation (60, 65, & 70 dB) and the delay period between presentation of the first tonal pattern and presentation of the second comparison pattern (2 s, 3 s, or 4 s). As predicted, a significant interaction between delay time and presentation level was observed with participants making more errors and requiring greater time to respond in the 4 s delay condition, particularly in the lower 60 dB presentation level condition. Population aging and associated increases in the incidence of hearing loss are occurring at a time when verbal displays are increasingly being used in complex environments. Additionally, the need for persons of diverse language backgrounds to communicate in occupational environments with less than optimum acoustical environments make design accommodations that facilitate cognitive processing of speech in complex systems imperative. Slight increases in presentation level may be one way to facilitate the communication process by increasing the duration during which verbal material may be accessed for processing.