Search

Closed-circuit television (CCTV) is increasingly used as a means to ensure the safety and security of critical infrastructure and public spaces. Operators in control rooms are responsible for monitoring multiple camera feeds that generally exceed the number of screens available. Using a realistic video surveillance simulation, the current study investigates strategies that untrained operators use to deal with this visual overload. The majority of participants adopted a priority strategy by fixating some scenes more than others, as opposed to a parity strategy of devoting roughly equal time across all screens—although participants were largely unaware of the strategy they had used. A parity approach led to better detection performance, perhaps because less time elapsed between viewing each available camera feed thus reducing the probability that an incident could pass unnoticed. The identification of successful scanning strategies can be used to inform operator training.

The failure to notice critical changes in both visual and auditory scenes may have important consequences for performance in complex dynamic environments, especially those related to security, such as aviation, surveillance during major events, and command and control of emergency response. Previous work has shown that a significant number of situation changes remain undetected by operators in such environments. In the current study, we examined the impact of using auditory warning messages to support the detection of critical situation changes and to a broader extent the decision making required by the environment. Twenty-two participants performed a radar operator task involving multiple subtasks while detecting critical task-related events that were cued by a specific type of audio message. Results showed that about 22% of the critical changes remained undetected by participants, a percentage similar to that found in previous work using visual cues to support change detection. However, we found that audio messages tended to bias threat evaluation toward perceiving objects as more threatening than they were in reality. Such findings revealed both benefits and costs associated with using audio messages to support change detection in complex dynamic environments.

While the development of intelligent technologies in security surveillance can augment human capabilities, they do not replace the role of the operator entirely; as such, when developing surveillance support it is critical that limitations to the cognitive system are taken into account. The current article reviews the cognitive challenges associated with the task of a CCTV operator: visual search and cognitive/perceptual overload, attentional failures, vulnerability to distraction, and decision-making in a dynamically evolving environment. While not directly applied to surveillance issues, we suggest that the NSEEV (noticing – salience, effort, expectancy, value) model of attention could provide a useful theoretical basis for understanding the challenges faced in detection and monitoring tasks. Having identified cognitive limitations of the human operator, this review sets out a research agenda for further understanding the cognitive functioning related to surveillance, and highlights the need to consider the human element at the design stage when developing technological solutions to security surveillance.

Dynamic and complex command and control situations often require the timely recognition of changes in the environment in order to detect potentially malicious actions. Change detection can be challenging within a continually evolving scene, and particularly under multitasking conditions whereby attention is necessarily divided between several subtasks. On-screen tools can assist with detection (e.g., providing a visual record of changes, ensuring that none are overlooked), however, in a high workload environment, this may result in information overload to the detriment of the primary task. One alternative is to exploit the auditory modality as a means to support visual change detection. In the current study, we use a naval air-warfare simulation, and introduce an auditory alarm to coincide with critical visual changes (in aircraft speed/direction) on the radar. We found that participants detected a greater percentage of visual changes and were significantly quicker to detect these changes when they were accompanied by an auditory alarm than when they were not. Furthermore, participants reported that mental demand was lower in the auditory alarm condition, and this was reflected in reduced classification omissions on the primary task. Results are discussed in relation to Wickens’ multiple resource theory of attention and indicate the potential for using the auditory modality to facilitate visual change detection.

This paper investigates the use of physio-behavioural detection of fun to model players’ preferences in real-time in the context of an adaptive game. To do so, a Physiological and Behavioural Model of Fun (PBMF), previously trained on 218 players, was used to model players’ preferences based n gameplay events. As a proof-of-concept, we leverged the PBMF to generate a simple player’s preference profile tailored to our test-bench game: Assassin’s Creed: Odyssey, an open-world, action-adventure game. This model associated every player to one of 3 predetermined stereotypical types of player, namely Fight, Stealth and Explore, which are closely tied to mechanics of the Assassin’s Creed series. Using the inferred preferences, we compared an adaptive vs a non-adaptive version of the same game and tested whether the adaptive version was perceived as more fun than the non-adaptive version by the 39 participants of this study. The results point to the creation of an accurate player’s preference profiles during a baseline mission, with profile matching both a “ground truth” Fun Trace – a continuous, subjective rating of a player’s fun – and a self-reported profile with an accuracy of \(69\%\) and \(72\%\) respectively. This, however, did not translate into a measurable difference in reported fun between the adaptive version of the game and the non-adaptive version in neither Fun Trace ratings nor questionnaire answers. Theses findings support that stereotypical preference modelling can be achieved successfully through a physio-behavioural model of fun, but that further investigation on adaptation strategies to those preferences are needed in order to reach the adaptive game’s promise of maximizing player’s enjoyment.

Finding a physiological signature of a player’s fun is a goal yet to be achieved in the field of adaptive gaming. The research presented in this paper tackles this issue by gathering physiological, behavioural and self-report data from over 200 participants who played off-the-shelf video games from the Assassin’s Creed series within a minimally invasive laboratory environment. By leveraging machine learning techniques the prediction of the player’s fun from its physiological and behavioural markers becomes a possibility. They provide clues as to which signals are the most relevant in establishing a physiological signature of the fun factor by providing an important score based on the predictive power of each signal. Identifying those markers and their impact will prove crucial in the development of adaptive video games. Adaptive games tailor their gameplay to the affective state of a player in order to deliver the optimal gaming experience. Indeed, an adaptive video game needs a continuous reading of the fun level to be able to respond to these changing fun levels in real time. While the predictive power of the presented classifier remains limited with a gain in the F1 score of 15% against random chance, it brings insight as to which physiological features might be the most informative for further analysis and discuss means by which low accuracy classification could still improve gaming experience.

While the usefulness of games extends beyond their entertainment value, the act of playing a game remains essentially tied to its positive experience. Techniques to assess the player’s experience have greatly improved in the past decade, yet several challenges remain such as identifying objective and dynamic measures that reflect the player’s emotions during the game. In this paper, we describe an innovative approach to capture the player’s experience that relies on cognitive sciences and affective neuroscience. Our research endeavor is to contribute to the development of systems capable of predicting the player’s fun based on psychophysiology and in-game behaviors, and adapting the game to maximize that value. We present a use case of our techniques to elicit the player’s affective and cognitive states using an online strategic card game. Preliminary results revealed that electrodermal and respiratory activities were positively associated to the casual gamers’ affective and cognitive states. Such findings suggest that psychophysiological metrics combined with behavioural measures offer a promising avenue to assess the player’s experience in a comprehensive and objective manner.

Interindividual variability in the emotional expressivity of gamers is an issue in the development of emotion-based adaptive games. Specifically, the emotional expressivity of expert gamers has been shown to be reduced compared to non experts in previous work, but these observations have not been studied during a gaming session. This study tested whether expert gamers have a reduced emotional expressivity compared to non experts in the ecological context of a gaming session. Nine expert and nine non expert gamers aged between 18 and 35 years old played two missions of the game Assassin’s Creed: Syndicate. The muscular activity of the zygomaticus major and the corrugator supercilii was measured using facial electromyography. The subjective workload of the gaming session was assessed with the NASA-TLX questionnaire. Preliminary results suggest no difference between expert and non expert gamers on emotional expressivity as measured through facial electromyography. However, experts perceived their workload as less mentally and physically challenging compared to that of non experts. These results highlight the importance of considering a gamer’s level of expertise in emotion-based adaptive games.

Surveillance operators must scan multiple camera feeds to ensure timely detection of incidents; however, variability in scanning behavior can lead to untimely/failed detection of critical information in feeds that were neglected for a long period. Using an eye tracker to monitor screen fixations we can calculate (in real-time) the time elapsed since the last scan of each particular feed, allowing the setting-up of targeted countermeasures contingent on operator oculomotor behavior. One avenue is to provide operators with timely alerts to modulate the scan pattern to avoid attentional tunneling and inattentional blindness. We test such an adaptive solution within a major event surveillance simulation and preliminary results show that operator scan behavior can be modulated, although further investigation is required to determine warning frequency and modality to optimize the balance between saliency and workload increase. Future work will focus on adding a real-time vigilance detection and countermeasure capability.

There is ample evidence that reading speed increases when participants read the same text more than once. However, less is known about the impact of text repetition as a function of word class. Some authors suggested that text repetition would mostly benefit content words with little or no effect on function words. In the present study, we examined the effect of multiple readings on the processing of content and function words. Participants were asked to read a short text two times in direct succession. Eye movement analyses revealed the typical multiple readings effect: Repetition decreased the time readers spent fixating words and the probability of fixating critical words. Most importantly, we found that the effect of multiple readings was of the same magnitude for content and function words, and for low- and high-frequency words. Such findings suggest that lexical variables have additive effects on eye movement measures in reading.Keywords: text repetition, reading, word class, word frequency, eye movementsWhen readers read the same text more than once, they become faster at reading it (e.g., Raney & Rayner, 1995; Raney, Therriault, & Minkoff, 2010; Levy, Di Persio, & Hollingshead, 1992; Levy, Masson, & Zoubek, 1991). This phenomenon, known as the text repetition effect or the multiple readings effect, is particularly of interest since rereading is a common behavior, especially used to learn new information. Moreover, the study of text repetition provides an avenue for exploring the effect of text difficulty on the cognitive processes involved in reading. Although the multiple readings effect has been extensively investigated over the past decades, only a few studies have examined how text repetition influences the processing of individual words. In the present study, we tested this issue by contrasting eye movement measures for content and function words in two successive readings of the same text.Linguistic distinctions between function and content words are found in all languages and are thought to play a critical role in the acquisition of language (Shi, Werker, & Morgan, 1999). Function words provide grammatical relations between content words and consist of closed-class words that are primarily structural such as articles, prepositions, and auxiliaries. Most of the time, they include only one syllable and are very frequent in natural language (e.g., the, of, and, to). In contrast, content words convey the meaning of a sentence and correspond to open-class words such as nouns, verbs, adverbs, and adjectives. They are usually longer than function words and are less frequent in natural language. The linguistic distinctions between content and function words have been extensively studied. However, as pointed out by Schmauder, Morris, and Poynor (2000), the extent to which linguistic distinctions translate into psychological representations and processing distinctions has received much less attention.A number of studies have shown that content and function words might be processed differently during normal reading (e.g., Carpenter & Just, 1983; Drieghe, Pollatsek, Staub, & Rayner, 2008; Gauthier, O'Regan, & Le Gargasson, 2000; O'Regan, 1979; Schmauder et al., 2000). Using eye movements as a measure of cognitive processes involved in reading, those studies revealed that function words are less likely to be fixated than content words (but see Schmauder et al., 2000), a phenomenon called the "the-skipping effect" (e.g., Gauthier et al., 2000; O'Regan, 1979) or the word class effect (e.g., Levy, 1983; Roy-Charland, Saint-Aubin, Klein, & Lawrence, 2007). For instance, Carpenter and Just (1983) found that among three-letter words, function words received a lower proportion of fixation (.40) than content words (.57). This finding has been replicated recently by Drieghe et al., who showed that the French article "les" was skipped much more than three-letter verbs (see also Gauthier et al. …

The key to the development of adaptive gameplay is the capability to monitor and predict in real time the players experience (or, herein, fun factor). To achieve this goal, we rely on biometrics and machine learning algorithms to capture a physiological signature that reflects the player's affective state during the game. In this paper, we report research and development effort into the real time monitoring of the player's level of fun during a commercially available video game session using physiological signals. The use of a triple-classifier system allows the transformation of players' physiological responses and their fluctuation into a single yet multifaceted measure of fun, using a non-linear gameplay. Our results suggest that cardiac and respiratory activities provide the best predictive power. Moreover, the level of performance reached when classifying the level of fun (70% accuracy) shows that the use of machine learning approaches with physiological measures can contribute to predicting players experience in an objective manner.

The intensive care unit (ICU) is a high-risk environment that requires cross-professional teams to provide life-saving patient care. There is ample evidence that poor communication creates situations where medical errors are likely to occur and affect patient safety. We tested whether communication-oriented debriefing following high-fidelity simulation improves quality of information exchange reflecting collaborative work in ICU teams. Ten teams of six cross-professional ICU workers participated in three simulation-based training sessions. After each training session, the experimental group was debriefed on communication-oriented skills (based on Crew Resource Management, CRM), while the control group was debriefed on technical skills. The analysis was double-blind; 30 videotaped sessions were coded for three types of communication measures by four observers showing adequate inter-rater reliability. Results suggest that high-fidelity simulation training can improve non-technical skills in cross-professional ICU teams. Further investigation is needed on the performance effects of communication-focused debriefing.

Task switching research has so far focused on the impact of switching task-sets between two-choice classification tasks that require little or no memory load. Empirical work is lacking however to determine whether the switching cost can be extended to other cognitive activities and to different types of switches. In the present study, switching between the content - verbal to spatial - of the tasks was contrasted with switching cognitive processes - categorization to serial memory. Our pattern of results revealed the absence of local and general switch costs on serial memory tasks, while substantial costs were observed with two-choice judgement tasks. Such a finding challenges the widely accepted assumption that task alternation comes with a considerable cost in performance regardless of the cognitive tasks undertaken. Our results are discussed in context of the predominant models of task switching.

In the present study we examined the contribution of semantic associative links to short-term recall performance by using the separation effect first introduced in free recall studies (Glanzer, 1969). Pairs of associated words were inserted in the to-be-remembered lists. In two experiments associated words were better recalled than non-associated words, and were better recalled when they were adjacent in the list than when they were separated by one non-associated item. In addition results showed that forward associative links among pair members were as beneficial to immediate serial recall as backward associative links. Finally the benefit of associative links among pair members was observed with both forward and backward recall.

When participants are asked to recall lists of items in the reverse order, known as backward recall, several benchmark memory phenomena, such as the word length effect, are abolished (Bireta et al. Memory & Cognition 38:279–291, 2010). Bireta et al. (Memory & Cognition 38:279–291, 2010) suggested that in backward recall, reliance on order retention is increased at the expense of item retention, leading to the abolition of item-based phenomena. In a subsequent study, however, Guerard and Saint-Aubin (in press) showed that four lexical factors known to modulate item retention were unaffected by recall direction. In a series of five experiments, we examined the source of the discrepancy between the two studies. We revisited the effects of phonological similarity, word length, articulatory suppression, and irrelevant speech, using open and closed pools of words in backward and forward recall. The results are unequivocal in showing that none of these effects are influenced by recall direction, suggesting that Bireta et al.’s (Memory & Cognition 38:279–291, 2010) results are the consequence of their particular stimuli.

Temporal awareness is pivotal to successful real-time dynamic decision making in a wide range of command and control situations; particularly in safety-critical environments. However, little explicit support for operators' temporal awareness is provided by decision support systems (DSS) for time-critical decisions. In the context of functional simulations of naval anti-air warfare and emergency response management, the present study compares operator support provided by two display formats. In both environments, we contrast a baseline condition to a condition in which a temporal display was integrated to the original interface to support operators' temporal awareness. We also wish to establish whether the implementation of time-based DSSs may also come with drawbacks on cognitive functioning and performance.