Search

Consumer neurotechnology is arriving en masse, even while algorithms for user state estimation are being actively defned and developed. Indeed, many consumable wearables are now available that try to estimate cognitive changes from wrist data or body movement. But does this data help people? It’s a critical time to ask how users could be informed by wearable neurotechnology, in a way that would be relevant to their needs and serve their personal well-being. The aim of this SIG is to bring together the key HCI communities needed to address this: personal informatics, digital health and wellbeing, neuroergonomics, and neuroethics.

The motivation behind using physiological measures to estimate cognitive activity is typically to build technology that can help people to understand themselves and their work, or indeed for systems to do so and adapt. While functional Near Infrared Spectroscopy (fNIRS) has been shown to reliably reflect manipulations of mental workload in different work tasks, we still need to establish whether fNIRS can differentiate variety within common office-like tasks in order to broaden our understanding of the factors involved in tracking them in real working conditions. 20 healthy participants (8 females, 12 males), whose work included office-like tasks, took part in a user study that investigated a) the sensitivity of fNIRS for measuring mental workload variations in representations of everyday reading and writing tasks, and b) how representations of natural interruptions are reflected in the data. Results supported fNIRS measuring PFC activation in differentiating between workload levels for reading tasks but not writing tasks in terms of increased oxygenated haemoglobin (O2Hb) and decreased deoxygenated haemoglobin (HHb), for harder conditions compared to easier conditions. There was considerable support for fNIRS in detecting changes in workload levels due to interruptions. Variations in workload levels during the interruptions could be understood in relation to spare capacity models. These findings may guide future work into sustained monitoring of cognitive activity in real-world settings.

Functional Near-Infrared Spectroscopy (fNIRS) has shown promise for being potentially more suitable (than e.g. EEG) for brain-based Human Computer Interaction (HCI). While some machine learning approaches have been used in prior HCI work, this paper explores different approaches and configurations for classifying Mental Workload (MWL) from a continuous HCI task, to identify and understand potential limitations and data processing decisions. In particular, we investigate three overall approaches: a logistic regression method, a supervised shallow method (SVM), and a supervised deep learning method (CNN). We examine personalised and generalised models, as well as consider different features and ways of labelling the data. Our initial explorations show that generalised models can perform as well as personalised ones and that deep learning can be a suitable approach for medium size datasets. To provide additional practical advice for future brain-computer interaction systems, we conclude by discussing the limitations and data-preparation needs of different machine learning approaches. We also make recommendations for avenues of future work that are most promising for the machine learning of fNIRS data.

In the overlap between Human-Computer Interaction (HCI) and Cinematics, sits an interest in physiological responses to experiences. Focusing particularly on brain data, Neurocinematics has emerged as a research field using Brain-Computer Interface (BCI) sensors. Where previous work found inter subject correlations (ISC) between brain measurements of people watching movies in constrained conditions using functional magnetic resonance imaging (fMRI), we seek to examine similar responses in more naturalistic settings using functional Near Infrared Spectroscopy (fNIRS). fNIRS has been shown to be highly suitable for HCI studies, being more portable than fMRI and more tolerant of many natural movements than Electroencephalography (EEG). Early results found significant ISC, which gives a lot of hope and potential for using fNIRS in Neurocinematics.

Recent work has demonstrated that functional Near-Infrared Spectroscopy has the potential to measure changes in Mental Workload with increasing ecological validity. It is not clear, however, whether these measurements are affected by anxiety and stress of the workload, where our informal observations see some participants enjoying the workload and succeeding in tasks, while others worry and struggle with the tasks. This research evaluated the effects of stress on fNIRS measurements and performance, using the Montreal Imaging Stress Task to manipulate the experience of stress. While our results largely support this hypothesis, our conclusions were undermined by data from the Rest condition, which indicated that Mental Workload and Stress were often higher than during tasks. We hypothesize that participants were experiencing anxiety in anticipation of subsequent stress tasks. We discuss this hypothesis and present a revised study designed to better control for this result.

Amongst the many tasks in our lives, we encounter web forms on a regular basis, whether they are mundane like registering for a website, or complex and important like tax returns. There are many aspects of Usability, but one concern for user interfaces is to reduce mental workload and error rates. Whilst most assessment of mental workload is subjective and retrospective reporting by users, we examine the potential of functional Near Infrared Spectroscopy (fNIRS) as a tool for objectively and concurrently measuring mental workload during usability testing. We use this technology to evaluate the design of three different form layouts for a car insurance claim process, and show that a form divided into subforms increases mental workload, contrary to our expectations. We conclude that fNIRS is highly suitable for objectively examining mental workload during usability testing, and will therefore be able to provide more detailed insight than summative retrospective assessments. Further, for the fNIRS community, we show that the technology can easily move beyond typical psychology tasks, and be used for more natural study tasks.

Recent efforts have shown that functional near-infrared spectroscopy (fNIRS) has potential value for brain sensing in HCI user studies. Research has shown that, although large head movement significantly affects fNIRS data, typical keyboard use, mouse movement, and non-task-related verbalisations do not affect measurements during Verbal tasks. This work aims to examine the Reliability of fNIRS, by 1) confirming these prior findings, and 2) significantly extending our understanding of how artefacts affect recordings during Spatial tasks, since much of user interfaces and interaction is inherently spatial. Our results show that artefacts have a significantly different impact during Verbal and Spatial tasks. We contribute clearer insights into using fNIRS as a tool within HCI user studies.

The Think Aloud Protocol (TAP) is a verbalisation technique widely employed in HCI user studies to give insight into user experience, yet little work has explored the impact that TAPs have on participants during user studies. This paper utilises a brain sensing technique, fNIRS, to observe the effect that TAPs have on participants. Functional Near-Infrared Spectroscopy (fNIRS) is a brain sensing technology that offers the potential to provide continuous, detailed insight into brain activity, enabling an objective view of cognitive processes during complex tasks. Participants were asked to perform a mathematical task under 4 conditions: nonsense verbalisations, passive concurrent think aloud protocol, invasive concurrent think aloud protocol, and a baseline of silence. Subjective ratings and performance measures were collected during the study. Our results provide a novel view into the effect that different forms of verbalisation have on workload during tasks. Further, the results provide a means for estimating the effect of spoken artefacts when measuring workload, which is another step towards our goal of proactively involving fNIRS analysis in ecologically valid user studies.