Search

Haptic technologies are widely used in multimedia entertainment to psychophysiologically enhance user experience. Psychometric-based research regarding vibrokinetic stimulation during multimedia viewing supports this notion. However, scant neurophysiological evidence exists to verify this effect. Using a between groups design with source-localized electroencephalography, the present study analyzed the effect of high fidelity vibrokinetic (HFVK) stimulation during passive multimedia interaction (i.e. watching a haptically enhanced movie) on self-reported emotional state and intercortical theta coherence. Results indicate that HFVK increases emotional reactivity in association with increased interhemispheric coherence between the right inferiortemporal gyrus and the left insular cortex, thereby conferring neurophysiological support for the efficaciousness of HFVK to enhance emotional response during movie watching.

Because of the COVID-19 pandemic, telework policies have required many user experience (UX) labs to restrict their research activities to remote user testing. Automatic Facial Expression Analysis (AFEA) is an accessible psychophysiological measurement that can be easily implemented in remote user tests. However, to date, the literature on Human Computer Interaction (HCI) has provided no guidelines for remote moderated user tests that collect facial expression data and synchronize them with the state of a dynamic stimulus such as a webpage. To address this research gap, this article offers guidelines for effective AFEA data collection that are based on a methodology developed in a concrete research context and on the lessons learned from applying it in four remote moderated user testing projects. Since researchers have less control over test environment settings, we maintain that they should pay greater attention to factors that can affect face detection and\or emotion classification prior, during, and after remote moderated user tests. Our study contributes to the development of methods for including psychophysiological and neurophysiological measurements in remote user tests that offer promising opportunities for information systems (IS) research, UX design, and even digital health research.

The development of assistive technologies and guidelines for their accessibility is impeded by the limited access to disabled participants. As a consequence, performing disability simulations on able bodied participants is a common practice in usability evaluation of assistive technologies. However, still little is known about how disability simulation can influence the usability evaluation of assistive technologies by able-bodied participants. This research proposal explores the effect of a motor dysfunction simulation in able-bodied participants that impedes use of a mouse input, but supports gesture-based assistive technology. Results of this study may provide insights on how to improve, via the experimental design, the meaning and the validity of usability evaluation of assistive technologies by able-bodied participants.

Hedonic videos are often accompanied by instrumental text/subtitles. However, the neurophysiological effects of task-switching between instrumental text processing and audiovisual processing during hedonic video watching remains unexplored. We investigated these effects using changes in pupil diameter and theta-band spectral power as indices of cognitive load, and self-reported and automated facial expression-based measures of emotion. We found that the presence of instrumental subtitles subjectively improved comprehension without negatively impacting cognitive and emotional response. These findings contribute to the literature by showing that in this specific context, multitasking does enhance the user experience. Further research is needed to explore the extent to which instrumental text is useful, and how its effects change in accordance with audio presence during passive hedonic experiences across various contexts.

In the last decade, entertainment industries have been creating multi-sensory experiences using haptic stimulation delivered via vibro-kinetic (VK) seats that produce vibration and motion coincidental with audiovisual content. However, the effects of VK stimulation during purely auditory experiences remains unclarified. We addressed this scientific shortcoming by investigating the effects of high fidelity VK (HFVK) stimulation, on psychophysiological indices of emotional valence and arousal. The HFVK stimulation was delivered to 24 healthy participants through an HFVK enhanced seat during active music listening. Participants listened to six popular songs randomly chosen from a pool of 15 songs, with the songs presented randomly with or without HFVK stimulation. Psychological emotional valence and arousal were indexed based on two items of the Self-Assessment Manikin (SAM) scale. Physiological emotional valence and arousal were respectively indexed based on micro facial expressions and electrodermal activity. Our results revealed that psychological and physiological valence, and psychological arousal were higher with HFVK stimulation compared to a control condition where the chair was static. A post-experiment questionnaire further revealed greater subjective appreciation for the HFVK stimulation condition compared to the control condition. Overall, our results highlight the potential for HFVK technologies to enhance auditory listening experiences.

Objective.High-fidelity vibrokinetic (HFVK) technology is widely used to enhance the immersiveness of audiovisual (AV) entertainment experiences. However, despite evidence that HFVK technology does subjectively enhance AV immersion, the underlying mechanism has not been clarified. Neurophysiological studies could provide important evidence to illuminate this mechanism, thereby benefiting HFVK stimulus design, and facilitating expansion of HFVK technology.Approach.We conducted a between-subjects (VK,N= 11; Control,N= 9) exploratory study to measure the effect of HFVK stimulation through an HFVK seat on electroencephalographic cortical activity during an AV cinematic experience. Subjective appreciation of the experience was assessed and incorporated into statistical models exploring the effects of HFVK stimulation across cortical brain areas. We separately analyzed alpha-band (8–12 Hz) and theta-band (5–7 Hz) activities as indices of engagement and sensory processing, respectively. We also performed theta-band (5–7 Hz) coherence analyses using cortical seed areas identified from the theta activity analysis.Main results.The right fusiform gyrus, inferiotemporal gyrus, and supramarginal gyrus, known for emotion, AV-spatial, and vestibular processing, were identified as seeds from theta analyses. Coherence from these areas was uniformly enhanced in HFVK subjects in right motor areas, albeit predominantly in those who were appreciative. Meanwhile, compared to control subjects, HFVK subjects exhibited uniform interhemispheric decoherence with the left insula, which is important for self-processing.Significance.The results collectively point to sustained decoherence between sensory and self-processing as a possible mechanism for how HFVK increases immersion, and that coordination of emotional, spatial, and vestibular processing hubs with the motor system may be required for appreciation of the HFVK-enhanced experience. Overall, this study offers the first ever demonstration that HFVK stimulation has a real and sustained effect on brain activity during a cinematic experience.