Search

The Low/High Index of Pupillary Activity (LHIPA), an eye-tracked measure of pupil diameter oscillation, is redesigned and implemented to function in real-time. The novel Real-time IPA (RIPA) is shown to discriminate cognitive load in re-streamed data from earlier experiments. Rationale for the RIPA is tied to the functioning of the human autonomic nervous system yielding a hybrid measure based on the ratio of Low/High frequencies of pupil oscillation. The paper's contribution is drawn from provision of documentation of the calculation of the RIPA. As with the LHIPA, it is possible for researchers to apply this metric to their own experiments where a measure of cognitive load is of interest.

Subtitle production is an increasingly creative accessibility service. New technologies allow for placing subtitles at any location of the screen in a variety of formats, shapes, typography, font size, and colour. The screen now affords accessible creativity, with subtitles able to provide novel experiences beyond those offered by traditional language translation. Immersive environments multiply 2D subtitles features to new creative viewing modalities. Testing subtitles in eXtended Reality (XR) has pushed existing methods to address user need and enjoyment of audiovisual content in 360º viewing displays. After an overview of existing subtitle features in XR, the article describes the challenges of generating subtitle stimuli to test meaningful user viewing behaviours, based on eye-tracking technology. The approach for the first experimental setup for implementing creative subtitles in XR using eye-tracking is given, in line with novel research questions. The choices made regarding sound, duration and storyboard are described. Conclusions show that testing subtitles in immersive media environments is both a linguistic and an artistic endeavour, which requires an agile framework fostering contrast and comparison of different functionalities. Results of the present, preliminary study shed light on future experimental setups with eye-tracking. Lay summary Subtitling is an increasingly creative accessibility service. Subtitlers use new technologies to place subtitles at any location on the screen. They can also change shape, typography, font size, and colour of the subtitles. This personalisation opens the door for creative solutions and allows novel experiences beyond traditional 2D subtitles. An immersive environment is an illusionary experience that surrounds you and transports you to another place. The challenges when testing subtitles in these new environments are different from those when testing in 2D media, as the user is no longer a passive spectator but an active part of the story. In this article, we give an overview of existing subtitle features in immersive environments. Then we describe the challenges of generating stimuli suitable for testing subtitles in immersive environments using eye tracking technology. We explain the experimental setup used during our experiment as well as the research questions. We describe the choices we made to design our stimuli, such as the sound, the duration, and the storyboard. We conclude that testing subtitles in immersive media environments present linguistics and artistic challenges and that we need to use a framework that allows for rapid contrast of different solutions. Results of the present, preliminary study clear up how to use eye-tracking technology to test subtitles in immersive environments.

Krejtz, K., Żurawska, J., Duchowski, A., & Wichary, S. (2020). Pupillary and microsaccadic responses to cognitive effort and emotional arousal during complex decision making.Journal of Eye Movement Research,13(5): 2. Abstract A large body of literature documents the sensitivity of pupil response to cognitive load (e.g., Krejtz et al. 2018) and emotional arousal (Bradley et al., 2008). Recent empirical evidence also showed that microsaccade characteristics and dynamics can be modulated by mental fatigue and cognitive load (e.g., Dalmaso et al. 2017). Very little is known about the sensitivity of microsaccadic characteristics to emotional arousal. The present paper demonstrates in a controlled experiment pupillary and microsaccadic responses to information processing during multi-attribute decision making under affective priming. Twenty-one psychology students were randomly assigned into three affective priming conditions (neutral, aversive, and erotic). Participants were tasked to make several discriminative decisions based on acquired cues. In line with the expectations, results showed microsaccadic rate inhibition and pupillary dilation depending on cognitive effort (number of acquired cues) prior to decision. These effects were moderated by affective priming. Aversive priming strengthened pupillary and microsaccadic response to information processing effort. In general, results suggest that pupillary response is more biased by affective priming than microsaccadic rate. The results are discussed in the light of neuropsychological mechanisms of pupillary and microsaccadic behavior generation.

Virtual Reality (VR) headsets can open opportunities for users to accomplish complex tasks on large virtual displays, using compact setups. However, interacting with large virtual displays using existing interaction techniques might cause fatigue, especially for precise manipulations, due to the lack of physical surfaces. We designed VXSlate, an interaction technique that uses a large virtual display, as an expansion of a tablet. VXSlate combines a user_s head movements, as tracked by the VR headset, and touch interaction on the tablet. The user_s head movements position both a virtual representation of the tablet and of the user_s hand on the large virtual display. The user_s multi-touch interactions perform finely-tuned content manipulations., 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

Eye motions constitute an important part of our daily face-to-face interactions. Even subtle details in the eyes’ motions give us clues about a person’s thoughts and emotions. Believable and natural animation of the eyes is therefore crucial when creating appealing virtual characters. In this article, we investigate the perceived naturalness of detailed eye motions, more specifically of jitter of the eyeball rotation and pupil diameter on three virtual characters with differing levels of realism. Participants watched stimuli with six scaling factors from 0 to 1 in increments of 0.2, varying eye rotation and pupil size jitter individually, and they had to indicate if they would like to increase or decrease the level of jitter to make the animation look more natural. Based on participants’ responses, we determine the scaling factors for noise attenuation perceived as most natural for each character when using motion-captured eye motions. We compute the corresponding average jitter amplitudes for the eyeball rotation and pupil size to serve as guidelines for other characters. We find that the amplitudes perceived as most natural depend on the character, with our character with a medium level of realism requiring the largest scaling factors.

In this study, we examined whether interpreters and interpreting trainees are better predisposed to respeaking than people with no interpreting skills. We tested 57 participants (22 interpreters, 23 translators and 12 controls) while respeaking 5-minute videos with two parameters: speech rate (fast/slow) and number of speakers (one/many). Having measured the quality of the respeaking performance using two independent methods: the NER model and rating, we found that interpreters consistently achieved higher scores than the other two groups. The findings are discussed in the context of transfer of skills, expert performance and respeaking training.

We discuss several techniques for mapping the gaze point to fiducial markers detected in a video stream, as commonly used in Augmented Reality eye-tracking applications. Specifically, we focus on using the recovered camera rotation and translation determined by the apparent distortion of the marker. We exploit this to map Areas Of Interest (AOIs) in the plane relative to the given marker. This gives the advantage of not needing any more than a single marker. We also review how the recovered homography is transformed to the graphics world coordinates so that AOI visualization can be performed in world space instead of camera space.

The enhancement hypothesis suggests that deaf individuals are more vigilant to visual emotional cues than hearing individuals. The present eye-tracking study examined ambient–focal visual attention when encoding affect from dynamically changing emotional facial expressions. Deaf (n = 17) and hearing (n = 17) individuals watched emotional facial expressions that in 10-s animations morphed from a neutral expression to one of happiness, sadness, or anger. The task was to recognize emotion as quickly as possible. Deaf participants tended to be faster than hearing participants in affect recognition, but the groups did not differ in accuracy. In general, happy faces were more accurately and more quickly recognized than faces expressing anger or sadness. Both groups demonstrated longer average fixation duration when recognizing happiness in comparison to anger and sadness. Deaf individuals directed their first fixations less often to the mouth region than the hearing group. During the last stages of emotion recognition, deaf participants exhibited more focal viewing of happy faces than negative faces. This pattern was not observed among hearing individuals. The analysis of visual gaze dynamics, switching between ambient and focal attention, was useful in studying the depth of cognitive processing of emotional information among deaf and hearing individuals.

Research has shown that keyframed eye motions are perceived as more realistic when some noise is added to eyeball motions and to pupil size changes. We investigate whether this noise, in contrast to being motion captured, can be synthesized with standard techniques, e.g., procedural or data-driven approaches. In a two-alternative forced choice task, we compare eye animations created with four different techniques: motion captured, procedural, data-driven, and keyframed (lacking noise). Our perceptual experiment uses three character models with different levels of realism and two motions. Our results suggest that procedural and data-driven noise can be used to create animations at similar perceived naturalness to our motion captured approach. Participants’ eye movements when viewing the animations show that animations without jitter yielded fewer fixations, suggesting ease of dismissal as unnatural.

A variety of psychological disorders like antisocial personality disorder have been linked to impairments in facial emotion recognition. Exploring eye movements during categorization of emotional faces is a promising approach with the potential to reveal possible differences in cognitive processes underlying these deficits. Based on this premise we investigated whether antisocial violent offenders exhibit different scan patterns compared to a matched healthy control group while categorizing emotional faces. Group differences were analyzed in terms of attention to the eyes, extent of exploration behavior and structure of switching patterns between Areas of Interest. While we were not able to show clear group differences, the present study is one of the first that demonstrates the feasibility and utility of incorporating recently developed eye movement metrics such as gaze transition entropy into clinical psychology.

In this paper we present preliminary results of the study on the cognitive load in intralingual and interlingual respeaking. We tested 57 subjects from three groups: interpreters, translators and controls while respeaking 5-minute videos in two language combinations: Polish to Polish (intralingual) and English to Polish (interlingual). Using two measures of cognitive load: self-report and EEG (Emotiv), we found that in most cases cognitive load was higher in interlingual respeaking. Self-reported mental effort that the participants had to expend to complete the respeaking tasks was lower in the group of interpreters, suggesting some parallels between interpreting and respeaking competences. EEG measures showed significant differences between respeaking tasks and experimental groups in cognitive load over time.

Szarkowska, Jankowska, Krejtz and Kowalski present an innovative approach towards creating accessible content following the example of a multimedia app created in the Open Art project, carried out by a consortium of partners including museums, universities, a research institute and a media access foundation. The goal of the project is to create a multimedia application guide containing accessible descriptions of selected works of Polish modern and contemporary art from the participating museums, in accordance with the principles of Universal Design. The chapter presents the concept of the app, focusing on the results of user-centred needs analysis and reception studies regarding the features of the optimum description of works of art in museums.