Search

In this study, a head-mounted camera was used to track eye behaviors and estimate the gaze point on the user’s visual plane. The integration of the elastic mechanism design makes the headset adaptable for various users. The wearable cases were prototyped with low-cost cameras to produce an efficient eye tracking solution. This proposed system can effectively extract and estimate pupil ellipse from a few camera images of an eye and compute the corresponding three-dimensional eye model. The system can match later images of the same pupil ellipse from a head-mounted camera to give the possible visual angles. To estimate the gaze point, the system uses multiple-point calibration to solve the related polynomial formula for future angle-to-gaze mapping. The proposed eye-tracking algorithms can provide a low-complexity solution with high accuracy, precision, and speed. This tracking system is a low-cost and promising system that can be used in headsets for virtual reality, auxiliary equipment, interactive machine, and human–machine interface applications. The proposed eye-tracking algorithm can achieve satisfactory performance without using a high-end high-speed camera and can be detected under different lighting sources, and the average errors of the detection results are stably within 9 pixels and at a distance of 50 cm from the screen; while the average error of the fixation mapping results is within 3°.

The outbreak of COVID-19 has led to worldwide quarantines and telehealth has become the lifeline for patients prone to infection. We propose using a Virtual Reality (VR) system as a playful coping strategy for rehabilitation at home. We hypothesize such a system can improve the physical and mental well-being of the user during play. In this demonstration, we transform the therapy into a playful virtual cat bathing simulation using a VR headset coupled with force feedback gloves. This results in an engaging and feedback-rich scenario where users practice fine motor skills by progressively completing three cat-care tasks.

This paper presents our reflection on adapting our methodology for developing training content for young jobseekers to new influencing factors. The development process centers around small development teams working incrementally to expand a suit of VR applications being used for career guidance, so-called ‘immersive job taste’. COVID-19 measures have already impacted our ability to carry out user evaluations. Our process can be negatively affected by a headset vendor’s new policy. This paper ‘s contribution is a perspective on the lessons learned and challenges of creating content for a set of VR-based training and career guidance apps targeted at young job seekers, as a part of Virtual Internship project.

The rapid emergence of augmented reality (AR) has brought considerable advantages to warehouse workers. However, due to inherent biological and cognitive differences, the male and female workers perceive cognitive loads differently. Understanding the differences is essential to improve the workers’ productivity and well-being. Therefore, we developed the AR headset that helped participants facilitate parcel scanning and evaluated the gender differences in the context of long-lasting repetitive parcel scanning. The results show that the female workers had significantly lower operational efficiency, higher visual attention, and higher memory loads than the male, but they quickly gained advantages in these aspects.

In this paper, we propose a potential cybersickness mitigation technique, redirected tilting, and conduct an exploratory virtual reality (VR) study to determine whether it is possible to visually induce head tilt during virtual locomotion. Redirected tilting involves rotating the virtual environment (VE) towards the turning direction around the VR headset’s roll axis, which we hypothesize could lead users to tilt their bodies in the same direction and maintain balance during curved paths. Unlike previous techniques, like field of view (FOV) restriction or rest frames, this method could potentially reduce cybersickness by manipulating egocentric self-motion rather than modifying visual characteristics of the VE. As an initial exploration, we conducted a within-subjects study with 30 participants to evaluate the effect of redirected tilting on postural behavior. The results showed that the proposed technique was successful in eliciting head tilt, although the magnitude of rotation was not as large as we had expected. We conclude that further investigation is needed to understand the mechanics, ideal parameters, and applications of redirected tilting.

With the rising popularity of Virtual Reality (VR), there is also a rising interest in co-located multiplayer experiences, as people want to play VR games together with their friends. As having multiple VR headsets is out of reach to the average consumer, we need to look into different possible ways of including multiple people in this play space. We have created a multi-modal co-located multiplayer VR game, Stuck in Space, that introduces a second player in two ways - one with a PC (the baseline that a lot of current games do), as well as a tracked Phone that can be used as a ‘window into the virtual world’. We have conducted a user study (n = 24) where we explore the difference in immersion and co-presence between the two versions using two questionnaires (IPQ and NMMoSP), as well as a thematic analysis of the subsequent interview data, from which 5 themes emerged. Surprisingly, we found no significant difference in co-presence or immersion based on the quantitative data. However, the qualitative analysis helps reveal one of the main reasons why that is - maintaining a mental model of the real world while also being in the virtual world makes it harder for the person wearing the headset to immerse themselves and feel co-present. From these themes and sub-themes we theorize that each of the two versions has positives and negatives that cancel each other out in the quantitative data, and for there to be a difference we would need to accentuate or change certain elements of the game. The results show that introducing a second player through a Phone is not detrimental in terms of co-presence and immersion and that it is a viable way of doing so, although certain design considerations would have to be taken into account to minimize the negatives.

We want to understand the changing shopping behaviour, influenced by health-targeting nutrition apps on mobile devices. To achieve that, we have built a realistic Virtual Reality (VR) supermarket simulation and addressed core aspects such as handling virtual products, prevention of cybersickness, rounded off with an initial pilot study. On top of that, we built a virtual replica smartphone in VR with nutritionrelated functionality. This has been extended with an Augmented Virtuality (AV) feature, that enables us to track the screen of a participant’s own smartphone, hence allowing us to integrate real-world apps and letting the user interact with them during the simulation. To achieve a high-quality tracking, we propose a hybrid approach, utilizing an add-on RGB camera on the VR headset fused with data provided through a WLAN connection in the case of self-made apps. This enables the users to manipulate the simulation from within the smartphone app, introducing a versatile, highly usability-centered controller because they can handle their phone naturally.

Interest in the use of virtual reality technologies for individuals with autism spectrum disorders has been increasing for over two decades. Recently, research interest has been growing in the area of head mounted display-based virtual reality technologies, thanks to increased availability and affordability. Affordances and theorized benefits of headset-based virtual reality for individuals with autism spectrum disorders are quite promising. However, very little attention has been given in the literature to implementation safety and ethics. This is a particular concern in light of documented adverse effects associated with headset-based virtual reality. To approach this gap, this article details how the authors approached the issue of minimizing adverse effects with related and overlapping methods, but from two separate, independent research sites—one in the United States and one in the United Kingdom. A structured within- and across-case analysis of the two independent studies was conducted to identify central implementation processes and procedures. Analysis resulted in development of a model for minimizing potential adverse effects of headset-based virtual reality for this population. We assert that our model could provide clarity in terms of design and implementation of headset-based virtual reality for individuals with autism spectrum disorders, guide implementations of future researchers and practitioners, and contribute to minimizing and controlling for potential adverse effects.

Augmented reality headsets have become increasingly consumer-available. Often gesture and speech are the main input modalities provided by these headsets. For some tasks, users may need a more precise input method. Tracked controllers can be added by using image tracking; however, this is not always the most accurate solution. This work outlines how to use off-the-shelf products to create a collaborative cross-device mixed reality experience. In that experience, the positionally tracked inputs from one headset can be used by another headset that may not natively support them.

In the near future, augmented reality (AR) glasses are envisioned to become the next-generation personal computing platform. They could be always on and worn all day, delivering continuous and pervasive AR experiences for general-purpose everyday use cases. However, it remains unclear how we could enable unobtrusive and easy information access without distracting users, while being acceptable to use at the same time. To address this question, we implemented two prototypes based on the Glanceable AR paradigm, a promising way of managing and acquiring information through glancing at the periphery of AR head-worn displays (HWDs). We conducted two separate studies to evaluate our designs. In the first study, we obtained feedback from a large sample of participants of varied age and background about a video prototype that showcased some envisioned scenarios of using Glanceable AR for everyday tasks. In the second study, we asked participants to use a working prototype during authentic real-world activities for three days. We found that users appreciated the Glanceable AR approach. They found it less distracting or intrusive than existing devices in authentic everyday use cases, and would like to use the interface on a daily basis if the form factor of the AR headset was more like eyeglasses.

Digital pen interaction has become a first-class input modality for precision tasks such as writing, annotating, drawing, and 2D manipulation. The key enablers of digital inking are the capacitive or resistive sensors that are integrated in contemporary tablet devices. In Virtual Reality (VR), however, users typically provide input across large regions, hence limiting the suitability of using additional tablet devices for accurate pen input. In this paper, we present Flashpen, a digital pen for VR whose sensing principle affords accurately digitizing hand writing and intricate drawing, including small and quick turns. Flashpen re-purposes an inexpensive gaming mouse sensor that digitizes extremely fine grained motions in the micrometer range at over 8 kHz when moving on a surface. We combine Flashpen's high-fidelity relative input with the absolute tracking cues from a VR headset to enable pen interaction across a variety of VR applications. In our two-block evaluation, which consists of a tracing task and a writing task, we compare Flashpen to a professional drawing tablet (Wacom). With this, we demonstrate that Flashpen's fidelity matches the performance of state-of-the-art digitizers and approaches the fidelity of analog pens, while adding the flexibility of supporting a wide range of flat surfaces.

Objective Late surgical correction of meningoencephalocele is a rare scenario that remains challenging for surgeons. Three-dimensional models can mimic the correct anatomical relationships, and technological systems have brought advances to medicine. This study aims to present a novel preoperative planning modality that combines augmented reality with a hybrid model for complex malformation associated with late correction. Methods The present report describes a case of frontoethmoidal encephalocele of a 10-year-old girl. Two different methods for planning the approach were developed: 1) a hybrid hands-on model and 2) augmented reality, using a cell phone app and headset display. The customized hybrid model was created based on patient's imaging. The augmented reality app was developed with a real-time interface. Results The hybrid model recreated anatomic alterations, thereby allowing a multidisciplinary team to determine an appropriate surgical approach. All aspects of the surgical procedure were simulated. A pre- to postoperative comparison was made, which emphasized benefit of tridimensional anatomical relationships using augmented reality tool and its role in preoperative planning. Conclusions The authors believe this new multimodality preoperative platform could be a useful method to improve multidisciplinary discussion as well as a powerful tool for teaching and planning.

The following research describes the potential of classifying a four-class emotion using a wearable EEG headset and using VR to induce emotional responses from the users. Various researchers have conducted emotion recognition using medical-grade EEG devices supported with a 2D monitor screen to induce emotional responses. This method of approach could cause additional artifacts due to the lack of concentration focusing within the border of the monitor screen of the intended stimulation thus reducing the classification accuracies. The large and complex EEG machine used by medical professions are sensitive equipment must be operated by trained professions thus making it difficult to seek permit to access such device. Hence, using a wearable EEG headset which is small and portable was considered for the brainwave signal samplings. this favors the researchers for use in conducting experiments for a human recognition system. The wearable EEG headset collects the brainwave signals at TP9, TP10, AF7, and AF8 electrode placements sampled at 256Hz with the five-bands (Delta, Theta, Alpha, Beta, Gamma). Additionally, the wearable EEG headset combines with the virtual reality (VR) headset to induce emotional responses presented to the users using the prepared VR video stimulus. The VR video was presented using the Arousal Valence Space (AVS) model with each of the respective quadrant having four videos presented in 80-seconds with a 10-second rest interval during transitions totaling up to 360-seconds from beginning to end. The collected samples are classified using Feedforward Artificial Neural Network (FANN) with 10-fold cross-validation and the model was trained using 90% of the total dataset with 10% used for validation purposes. The highest average classification result obtained from FANN was at 41.04%. While the classification performance was low, the confusion matrix presented a different view of the four-classes performed using different trained epoch values. Observations of trained epoch (2000, 3000, and 5000) showed each of the emotion classes happy, scared, bored, and calm, achieved classification accuracy of 75.15%, 75.12%, 75.02%, and 74.24% respectively.

We present HemodynamicVR, a virtual reality headset combined with functional near-infrared spectroscopy (fNIRS). We believe that sensing brain activity will enable novel interactions in virtual reality. In this paper, we assess a user’s cybersickness based on the change of their total hemoglobin concentration measured via an fNIRS device, and we try to mitigate that by changing the field of view (FOV) in real-time. In this experiment, participants experienced VR locomotion with an added variable FOV controlled by velocity changes and by fNIRS. The results suggest that fNIRS can detect cybersickness as registered by the qualitative SSQ test.

As post-secondary education migrates online, developing and evaluating new avenues for assessment in anatomy is paramount. Three-dimensional (3D) visualization technology is one area with the potential to augment or even replace resource-intensive cadaver use in anatomical education. This manuscript details the development of a smartphone application, entitled "Virtual Reality Bell-Ringer (VRBR)," capable of displaying monoscopic two-dimensional (2D) or stereoscopic 3D images with the use of an inexpensive cardboard headset for use in spot examinations. Cadaveric image use, creation, and pinning processes are explained, and the source code is provided. To validate this tool, this paper compares traditional laboratory-based spot examination assessment stations against those administered using the VRBR application to test anatomical knowledge. Participants (undergraduate, n = 38; graduate, n = 13) completed three spot examinations specific to their level of study, one in each of the modalities (2D, 3D, laboratory) as well as a mental rotation test (MRT), Stereo Fly stereotest, and cybersickness survey. Repeated measures ANCOVA suggested participants performed significantly better on laboratory and 3D stations compared to 2D stations. Moderate to severe cybersickness symptoms were reported by 63% of participants in at least one category while using the VRBR application. Highest reported symptoms included: eye strain, general discomfort, difficulty focusing, and difficulty concentrating. Overall, the VRBR application is a promising tool for its portability, affordability, and accessibility. Due to reported cybersickness and other technical limitations, the use of VRBR as an alternative to cadaveric specimens presents several challenges when testing anatomy knowledge that must be addressed before widespread adoption.

Virtual reality is a system that replaces the reality in a computer world and could be a game or a video that you play in a VR device like a VR headset. It is the utilization of PC innovation to establish a simulated environment to become a part and live that experience. The idea of virtual reality is to emerge into another world that is a virtual world. Virtual reality optimizes the user experience & is the way to meet and feel like a real-life community in the virtual world.Increased Reality (IR) is a technology that allows an individual to interact with a re-acting device regardless of whether this is a re-examination of today's reality or a conjured up environment. It is the way we perceive, feel and enter the past, the future and the present. The phase by which reality becomes a distorted reality is called VR. It could stretch from a PC game, from walking around our dream home, to walking around an untouchable universe. It could enter the globe. This paper provides a historical overview of virtual reality, list the basic terms and classes of VR systems, followed by entertainment applications of this technology. Typical VR systems are analyzed in a comprehensive manner. The entire implementation of VRs and their interrelationships: input devices, output devices and applications are extensively investigated. Furthermore, human factors and their consequences for the design problem of VE has been discussed.

Working memory is a core cognitive function and its deficits is one of the most common cognitive impairments. Reduced working memory capacity manifests as reduced accuracy in memory recall and prolonged speed of memory retrieval in older adults. Currently, the relationship between healthy older individuals’ age-related changes in resting brain oscillations and their working memory capacity is not clear. Eyes-closed resting electroencephalogram (rEEG) is gaining momentum as a potential neuromarker of mild cognitive impairments. Wearable and wireless EEG headset measuring key electrophysiological brain signals during rest and a working memory task was utilized. This research’s central hypothesis is that rEEG (e.g., eyes closed for 90 s) frequency and network features are surrogate markers for working memory capacity in healthy older adults. Forty-three older adults’ memory performance (accuracy and reaction times), brain oscillations during rest, and inter-channel magnitude-squared coherence during rest were analyzed. We report that individuals with a lower memory retrieval accuracy showed significantly increased alpha and beta oscillations over the right parietal site. Yet, faster working memory retrieval was significantly correlated with increased delta and theta band powers over the left parietal sites. In addition, significantly increased coherence between the left parietal site and the right frontal area is correlated with the faster speed in memory retrieval. The frontal and parietal dynamics of resting EEG is associated with the “accuracy and speed trade-off” during working memory in healthy older adults. Our results suggest that rEEG brain oscillations at local and distant neural circuits are surrogates of working memory retrieval’s accuracy and processing speed. Our current findings further indicate that rEEG frequency and coherence features recorded by wearable headsets and a brief resting and task protocol are potential biomarkers for working memory capacity. Additionally, wearable headsets are useful for fast screening of cognitive impairment risk.

Dynamic lighting is playing a key role in education, by considering the main photometric variables such as the correlated color temperature and the illuminance to increase student’s attention levels inside the classroom. In the case of design students, the project component is fundamental for teaching, where students mainly need to develop activities such as presentation to listen to the teacher instructions, ideation and sketching to propose a solution according to the problem being addressed, and exhibition to present the work that has been done. These activities require specific and adequate lighting conditions to generate a positive impact on the performance of students. This article presents the design of a dynamic lighting system capable of adjusting the correlated color temperature in a range from 2500 K to 6500 K and the illuminance levels in a range from 0 lx to 800 lx to enhance the sustained and fixed attention of design students inside the classroom according to the type of activity that is being developed. The performance of the system was evaluated experimentally by measuring student’s attention inside the ergonomics and usability laboratory by using the Gesell chamber, the Emotiv Epoc EEG Headset with 14 electrodes to measure the brain activity and obtain engagement and focus levels, the eye tracking Tobii glasses, and a protocol to evaluate performance including several surveys and camera observation. In conclusion, the dynamic lighting system can improve the attention of design students by configuring the photometric variables according to the type of activity that is being done.

We typically experience the world from a first-person perspective (1PP) but can sometimes experience events from a third-person perspective (3PP) much as an observer might see us. Little is known about how visual perspective influences the formation of memories for events. We developed an immersive virtual reality paradigm to examine how visual perspective during encoding influences memories. Across two studies, participants explored immersive virtual environments from first-person and third-person avatar perspectives while wearing an Oculus Rift headset. Memory was tested immediately (Study One and Study Two) and following a one-week delay (Study Two). We assessed the accuracy of visual memory using cued recall questions and spatial memory by asking participants to draw maps of the layout of each environment (Study One and Study Two). Additional phenomenological ratings were included to assess visual perspective during remembering (Study Two). There were no differences in the accuracy of visual information across the two studies, but 3PP experiences were found to increase spatial memory accuracy due to their wider camera field of view when compared to 1PP experiences. Our results also demonstrate that 3PP experiences create 3PP memories, as reflected by an increase in subjective ratings of observer-like perspectives during remembering. In sum, visual perspective during memory formation influences the accuracy of spatial but not visual information, and the vantage point of memories during remembering.

There are times when a person is in an environment where they must focus on one conversation when multiple others are happening around them. This is referred to as the cocktail party phenomenon. Individuals with impaired hearing lack this ability. This paper gives insight into how the brain handles these situations and how it filters out what a person is not focusing on. Three video monitors were placed in front of the subject and each source played video and audio. The objective was to find any changes in the accuracy of classification when video stimuli are provided. Using a g. Nautilus headset and multiple audio and video sources, EEG is collected from a subject. This data is collected for each data source. Each dataset is used to train a single machine learning classifier which distinguishes the source of sound with a certain accuracy. The results yield an average accuracy of 94.28%

BackgroundOtitis Media with Effusion (OME) causing hearing impairments affects ∼1 in 10 children starting school in UK/ Europe. 80% have at least one episode with most having conductive hearing loss. Studies showed children with OME hear better with bone conducting headsets. During COVID-19 we investigated whether children with deafness secondary OME, without access to audiology or grommet surgery, could be aided with bone conduction kits and the HearGlueEar app.MethodsStarting July 2020, during COVID-19, children aged 3-11 years with OME and on a grommet waiting list were invited to a single arm, prospective study. They received the kit, instructions and HearGlueEar app by post. By 3 weeks parents were asked to charge and pair the devices, attend a remote consultation and complete an OMQ-14 questionnaire. Remote follow-up lasted 3 months. Outcomes: ability to use the equipment, complete the questionnaire about child’s hearing and behaviour before and with the equipment, declining grommet surgery or where deafness resolved, and give opinion about the intervention.Findings26 children enrolled. Families used the kit at home and school. Most found remote consultations positive and convenient. OMQ-14 responses were 90% positive. Comments were: “Other people have said, wow his speech is clearer.”, “It is making a real difference at home.”, “He said over and over again, “I can hear everybody, wow, wow, wow.”, “It is no exaggeration to say this has made an astronomical improvement to his quality of life”. One child reported “I can hear my best friend again”. “She is getting on really well with the headphones - pairing them with the iPad at home is simply brilliant.” Three families continued with the headset to avoid grommets.InterpretationPosting a bone conduction kit, HearGlueEar app and remote consultation is effective support for children with deafness secondary to OME.FundingNone

Brain-computer interface (BCI) system aims to enable interaction with people and therefore the environment without muscular activation, using changes in brain signals due to the execution of cognitive tasks. The target of the presented work is to investigate the power of Emotiv EPOC + headset to detect and record the P300 wave. Moreover, the effect of preprocessing the acquired signal was studied. Five participants were asked to attend different sessions to an equivalent 6x6 matrix while the rows and columns were randomly flashed at a rate of 200 ms. The acquired EEG data were sent wirelessly to OpenViBE software, which is employed to run the P300 speller. Two classification methods were tried: Linear discriminate analysis (LDA) and support vector machine (SVM). The capability of the headset to detect the P300 signals is proven by the results. Additionally, results show that participants reached accuracy up to 90 and 70% after only two training sessions for Linear discriminate analysis (LDA) and support vector machine (SVM) classifiers, respectively. The significance of this work is to demonstrate that such a portable and affordable headset might be useful to design and implement a robust and reliable online P300-based BCI system.

Foreign objects may be present in the bladder due to deliberate insertion or displacement of the surrounding organs. There have been multiple cases of sharp objects such as nails, hair pins, and screws, as well as large objects such as AAA batteries, toothbrushes, and ballpoint pens found in the urethra and the bladder, which must have been aimed to induce sexual satisfaction associated with mental and psychic disorders. The diagnosis was established by means of radiological examinations, ie thoracic x-ray, KUB, and urethrography. This article reported a 32-year-old man with headset inserted to penile until bladder and could not be pulled out again. The patient complained pain and bleeding out of the urinary tract. We performed vesicotomy and pulling the out headset from bladder. The operation was succesful. In conclusion, the case of a headset in the bladder has been treated successfully with vesicotomy.

The objective of this short scientific communication is to describe and test a strategy to overcome communication barriers in coronavirus disease 2019 (COVID-19) era otolaryngology operating rooms. Thirteen endoscopic sinus surgeries, 4 skull base surgeries, and 1 tracheotomy were performed with powered air-purifying respirators. During these surgeries, surgical team members donned headsets with microphones linked via conference call. Noise measurements and survey responses were obtained and compared to pre-COVID-19 data. Noise was problematic and caused miscommunication as per 93% and 76% of respondents, respectively. Noise in COVID-19 era operating rooms was significantly higher compared to pre-COVID-19 era data (73.8 vs 70.2 decibels, P = .04). Implementation of this headset strategy significantly improved communication. Respondents with headsets were less likely to encounter communication problems (31% vs 93%, P < .001). Intraoperative measures to protect surgical team members during aerosolizing surgeries may impair communication. Linking team members via a conference call is a solution to improve communication.

Stress is a growing concern in this age of technological progress and productivity. Stress is traditionally measured using questionnaires with the help of psychologists. Due to shortage of health facilities and stigma associated with mental checkup, stress measurement process needs to be automated. This paper presents the results of an experiment to predict the perceived stress scores from electroencephalography (EEG) using a linear regression model. EEG signals are recorded using a five channel EEG headband for a duration of three minutes, with dry electrodes. In the first step, we acquire data by the Perceived Stress Scale (PSS) questionnaire and in the second step, we record EEG using muse headband and apply linear regression algorithm. The values of the PSS score serve as ground truth, which is then predicted by linear regression model. From the results, it is shown that the value of root mean squared error is as low as 2.36. This shows that PSS scores can be predicted with considerable accuracy by applying linear regression model on EEG signals. Our research suggests that perceived stress can be estimated from EEG signals using a low-channel headset once the system is trained.

This paper proposes a novel telemanipulation system that enables a human operator to control a dual-arm robot. The operation provides kinesthetic teaching via a digital twin of the robot which the operator cyber-physically guides to perform a task. Its key enabler is the concept of a virtual reality interactive marker, which serves as a simplified end effector of the digital twin robot. In virtual reality, the operator can interact with the marker using bare hands, which are sensed by the Leap Motion on top of a virtual reality headset. Then, the status (e.g. position/orientation) of the marker is transformed to the corresponding joint space command to the remote robot so that its end effector can follow the marker. We provide the details of the system architecture, and implement the system based on commercial robots/devices (i.e. UR5, Robotiq gripper, Leap Motion), virtual reality, ROS, and Unity3D. Moreover, the paper discusses the technical challenges that we had to address, and the system’s potential benefits from a human-robot interaction perspective.

In this paper, we study the task of facial expression recognition under strong occlusion. We are particularly interested in cases where 50% of the face is occluded, e.g. when the subject wears a Virtual Reality (VR) headset. While previous studies show that pre-training convolutional neural networks (CNNs) on fully-visible (non-occluded) faces improves the accuracy, we propose to employ knowledge distillation to achieve further improvements. First of all, we employ the classic teacher-student training strategy, in which the teacher is a CNN trained on fully-visible faces and the student is a CNN trained on occluded faces. Second of all, we propose a new approach for knowledge distillation based on triplet loss. During training, the goal is to reduce the distance between an anchor embedding, produced by a student CNN that takes occluded faces as input, and a positive embedding (from the same class as the anchor), produced by a teacher CNN trained on fully-visible faces, so that it becomes smaller than the distance between the anchor and a negative embedding (from a different class than the anchor), produced by the student CNN. Third of all, we propose to combine the distilled embeddings obtained through the classic teacher-student strategy and our novel teacher-student strategy based on triplet loss into a single embedding vector. We conduct experiments on two benchmarks, FER+ and AffectNet, with two CNN architectures, VGG-f and VGG-face, showing that knowledge distillation can bring significant improvements over the state-of-the-art methods designed for occluded faces in the VR setting., Accepted at ICPR 2020

This paper presents the development of a tool that allows people to communicate, by using only their voluntary eyes blinks. This tool provides a communication link mainly for people with motor disabilities, who cannot communicate through voice. the electroencephalographic (EEG) signal provided by the Mindwave Mobile 2 headset is the source of information to detect the voluntary blinks. In order to perform the digital processing of the EEG signal, a Long-Short Term Memory (LSTM) neural network was implemented, for the classification of the EEG signal in one of 5 possible classes: No blink, One blink, Two blinks, Three blinks and Other. the results of the trained model achieved an average accuracy of 92%. the neural network was embedded in a native Android mobile application, that shows a virtual keyboard consisting of the 27 letters of the Spanish alphabet, plus three characters, that are the commands “delete”, “space”, and “enter”. Each character can be selected by the user only through a determined number of voluntary blinks executed at certain times. When the user types a word and selects the “enter” command, the word is converted to audio using a text-to-speech engine. Finally, to verify the performance of the application, an experiment with eight people was executed, and as result the system achieved an average spelling precision of 91,26%.

Background Emotion prediction is a method that recognizes the human emotion derived from the subject’s psychological data. The problem in question is the limited use of heart rate (HR) as the prediction feature through the use of common classifiers such as Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Random Forest (RF) in emotion prediction. This paper aims to investigate whether HR signals can be utilized to classify four-class emotions using the emotion model from Russell’s in a virtual reality (VR) environment using machine learning. Method An experiment was conducted using the Empatica E4 wristband to acquire the participant’s HR, a VR headset as the display device for participants to view the 360° emotional videos, and the Empatica E4 real-time application was used during the experiment to extract and process the participant's recorded heart rate. Findings For intra-subject classification, all three classifiers SVM, KNN, and RF achieved 100% as the highest accuracy while inter-subject classification achieved 46.7% for SVM, 42.9% for KNN and 43.3% for RF. Conclusion The results demonstrate the potential of SVM, KNN and RF classifiers to classify HR as a feature to be used in emotion prediction in four distinct emotion classes in a virtual reality environment. The potential applications include interactive gaming, affective entertainment, and VR health rehabilitation.

This paper represents a rover design, which is controlled using human brain waves (EEG). Brain-computer interfacing (BCI) technology is followed for developing this rover. Neurosky mindwave mobile 2 headset is used for acquiring the desired brain data, which is wirelessly connected with the rover. The headset gives the output of attention level and number of eye blink based on EEG wave. The rover uses this attention level and eye blink to control the rover wheel and a 5 Degree of freedom (DOF) rover arm. This rover arm is implemented on the top of the rover body. The user controls the rover wheel and arm by changing the attention level and giving a different number of eye blinks simultaneously. This rover could be used to move an object from one place to another by controlling both rover arm and wheel portion. This rover has a real-time IoT based pulse monitoring system and Smoke detection system with GSM-based SMS alert.

Background: Otitis Media with Effusion (OME) causing hearing impairments affects ~1 in 10 children starting school in UK/ Europe. 80% have at least one episode with most having conductive hearing loss. Studies showed children with OME hear better with bone conducting headsets. During COVID-19 we investigated whether children with deafness secondary OME, without access to audiology or grommet surgery, could be aided with bone conduction kits and the HearGlueEar app. Methods: Starting July 2020, during COVID-19, children aged 3-11 years with OME and on a grommet waiting list were invited to a single arm, prospective study. They received the kit, instructions and HearGlueEar app by post. By 3 weeks parents were asked to charge and pair the devices, attend a remote consultation and complete an OMQ-14 questionnaire. Remote follow-up lasted 3 months. Outcomes: ability to use the equipment, complete the questionnaire about child’s hearing and behaviour before and with the equipment, declining grommet surgery or where deafness resolved, and give opinion about the intervention.Findings26 children enrolled. Families used the kit at home and school. Most found remote consultations positive and convenient. OMQ-14 responses were 90% positive. Comments were: “Other people have said, wow his speech is clearer.”, “It is making a real difference at home.”, “He said over and over again, “I can hear everybody, wow, wow, wow.”, “It is no exaggeration to say this has made an astronomical improvement to his quality of life”. One child reported “I can hear my best friend again”. “She is getting on really well with the headphones - pairing them with the iPad at home is simply brilliant.” Three families continued with the headset to avoid grommets. Interpretation: Posting a bone conduction kit, HearGlueEar app and remote consultation is effective support for children with deafness secondary to OME. Trial Registration: The study was registered IRAS 262816. Funding Statement: None of the authors have received any funding or payment for completing the research nor benefitted from sales of the equipment. Declaration of Interests: Dr Tamsin Holland Brown helped create the free, charity funded, Hear Glue Ear app that was used in this study. Dr Holland Brown has not received funding or payment related to the Hear Glue Ear app. Other authors declare no conflict of interest. Ethics Approval Statement: The study was approved by the Solihull REC and Health Research Authority (HRA).

In recent years, the surface defect detection technology of irregular industrial products based on machine vision has been widely used in various industrial scenarios. This paper takes Bluetooth headsets as an example, proposes a Bluetooth headset surface defect detection algorithm based on machine vision to quickly and accurately detect defects on the headset surface. After analyzing the surface characteristics and defect types of Bluetooth headsets, we proposed a surface scratch detection algorithm and a surface glue-overflowed detection algorithm. The result of the experiment shows that the detection algorithm can detect the surface defect of Bluetooth headsets fast as well as effectively, and the accuracy of defect recognition reaches 98%. The experiment verifies the correctness of the theory analysis and detection algorithm; therefore, the detection algorithm can be used in the recognition and detection of surface defect of Bluetooth headsets.

This study employs a novel emotion elicitation method to recollect EEG signals using an EEG-based BCI commercial device with 14 channels for emotion recognition with machine learning algorithms. We obtained EEG signals of six participants playing a complete Texas hold ’em poker game consisting of several hands in a sit-and-go online tournament with real money at stake. Poker was chosen because it is a game that provokes both positive and negative emotions associated with the sudden changes of the hands. To obtain the EEG readings, we use an EEG-based BCI headset with 14 electrodes, and the participants reported emotions based on self-assessment manikin in the valence – arousal space. This information was processed with kNN, Random Forest, and MLP neural network machine learning algorithms with 68.94, 68.58, and 68.82 values for accuracy, respectively. Therefore, the stimulus of poker game playing evoked emotions that were detectable and classifiable using automatic machine learning systems.

The association between personality and individual tendency for adopting and optimally interfacing with new technologies has often been proposed in the literature. However, only few published studies report experimental data. This study aims to provide evidence on the association between several individual variables and usability experience in modern immersive visual technologies. Our results are inconclusive regarding a relationship between participants’ personality and experienced sense of presence with perceived usability. The severity of the simulator sickness symptoms during the virtual experience is showed to be negatively associated with reported system usability. However, due to the small sample size and the number of variables examined, our results have only an explorative value. Qualitative analyses show that, despite positive attitude towards the use of VR for training, the ergonomic limitations of the VR headset and the suboptimal realism of the simulated scene were reported by the participants.

Advances in Augmented Reality technologies and, particularly, the availability of video see-through enabled head mounted displays (HMD), are allowing to devise new strategies to help individuals with visual impairments in daily life. In this work, an approach is proposed to compensate a serious visual impairment, known as metamorphopsia, a vision disorder characterized by deformed images. The goal is to provide patients with a digitally restored visual field, through real-time processing of video see-through streams captured from the HMD. To this regard, we present two contributions, respectively, an interactive discrete modeling of patient’s eye-specific vision distortion and a compensation of the latter by means of corresponding real-time counter-distortion of incoming frames. Our approach, indeed, maps each of the video streams acquired by the stereoscopic video see-through cameras aboard the headset on a 2D polygonal mesh which is then counter-warped by moving its vertices based on the previously built distortion model and then displayed, restored, on the HMD’s screen. First user evaluations report promising results along with usability issues related to HMD technology.

The use of Stereoscopic 3D (S3D) videos has been popular in commercial markets with ongoing developments in the field of visual entertainment in recent years. A wide variety of projection methods of 3D video content is currently available, such as projection to a panoramic screen and projection of omnidirectional video content from head mounted displays using Virtual Reality (VR) technology. This article investigates the Quality of Experience (QoE) and associated Visually Induced Motion Sickness (VIMS) caused by the viewing of S3D videos. The investigations used three different projection screens: a 3D flat screen, a 3D panoramic screen in a hemispherical shaped room and a VR headset. Several assessment methods including a Simulator Sickness Questionnaire (SSQ), ElectroEncephaloGraphy (EEG), and measurement tools for eye blink rate detection were applied to measure the QoE experienced by viewers. The SSQ scores were also compared with the behavioral data such as attention and meditation levels and enjoyment ratings acquired from different video content and projection screens. The results indicate that the projection screen is a key factor affecting the level of visual fatigue, VIMS and QoE assessments, which are discussed in-depth in the article.

The aim of this article is to propose a conceptual framework for a system, using which a severely ill or disabled patient admitted in a hospital cabin can exert some control on the surrounding environment using only his/her thoughts. A brain computer interface (BCI) headset can be used to collect Electroencephalographic (EEG) data from the brain and an artificial neural network can be used to process the EEG data and predict what the patient is thinking. The proposed system will provide immense benefits to the patients admitted in hospital cabins, especially in developing countries, where the doctor to patient and nurse to patient ratio is really low and a patient admitted in a hospital cabin is usually left unattended for a long period of time.

People who suffer from complete or partial paralysis often require the help of a second individual to move around from one place to another even with automated traveling mechanisms. This dependence on another person takes a toll on the individual. Thus, a framework is proposed wherein eye blinks and brain waves are used to decide the decisions made by the individual. This requires no extra physical exertion. These signals will be collected by an Electroencephalogram Headset (Neurosky Mindwave Mobile) and will then be passed to an android application which will then transfer necessary data to the Arduino, which will use these data to run the device. Such a framework can be executed on mechanized wheelchairs to uphold the impaired and help in their transition of becoming independent.

In this paper, the results of two pilot studies, carried out in a 3D classroom simulator (based on the Unity 3D engine), will be presented. A virtual classroom user uses a virtual reality headset, and teaches virtual learners (didactic-controlled), who report, ask, and answer questions, but also disturb. A total of 10 future geography teachers and 5 future computer science teachers participated in the research. In the virtual classroom, they underwent three-phase training, which was aimed at developing their pedagogical-psychological skills. In the first phase, they briefly tried working in the virtual classroom, so that they could prepare more accurately for the second phase, during which they were asked to explain the selected curriculum, or to assign virtual or individual work to virtual pupils. The performance in the virtual classroom was always followed by an in-depth analysis of the output with the facilitator and other students. A video recording of the whole performance was also made for students’ self-reflection. Students had the opportunity to reflect on their output from various points of view, and prepare for the last phase—repetition of the output in the virtual classroom (this time with the inclusion of comments, new approaches to the output, etc.). The research showed that reflective lessons in the virtual classroom have enormous potential for opening specific didactic topics during didactics lessons, both for the development of the study group discussion, and for deep reflection. In the third phase of the training, all of the students observed an improvement in a wide range of aspects of teaching. In the third phase of training, most students also experienced strong immersion—students get used to the virtual classroom, almost to the degree of forgetting where they are in the real world. This is all in spite the fact that the current prototype of the virtual classroom used is only at the beginning of its development, and will be gradually modified, complemented, and developed.

When it is tried to control drones, there are many different ways through various devices, using either motions like facial motion, special gloves with sensors, red, green, blue cameras on the laptop or even using smartwatches by performing gestures that are picked up by motion sensors. The paper proposes a work on how drones could be controlled using brainwaves without any of those devices. The drone control system of the current research was developed using electroencephalogram signals took by an Emotiv Insight headset. The electroencephalogram signals are collected from the users brain. The processed signal is then sent to the computer via Bluetooth. The headset employs Bluetooth Low Energy for wireless transmission. The brain of the user is trained in order to use the generated electroencephalogram data. The final signal is transmitted to Raspberry Pi zero via the MQTT messaging protocol. The Raspberry Pi controls the movement of the drone through the incoming signal from the headset. After years, brain control can replace many normal input sources like keyboards, touch screens or other traditional ways, so it enhances interactive experiences and provides new ways for disabled people to engage with their surroundings.

Measuring design performance for effective support of team coaching and redesign efforts has been difficult to near impossible. This is because, design is context dependent and takes place in different environments. Virtual reality gives us as designers, the opportunity to construct and simulate different environments, as coaches, the opportunity to improve our effectiveness in different design scenarios, and as researchers, the possibility to measure design performance and factors that affect it. This possibility was investigated experimentally. Two environments were constructed—one corresponding to a garage in a rural setting, and the other, to a conference room on one of the floors of a skyscraper in the city. Teams of three designers were recruited for the experiment. They worked on two product concept generation tasks and two decision making tasks, while being situated in each of the spaces. Several types of data were collected including video records, screen records, participant questionnaires, and position data of the VR headset and the hand controllers. The position data was used to calculate the level of synchrony between the designers. To investigate the correlation between the synchrony scores and the environment, we used a Kruskal-Wallis ordinal test. The test showed that the teams in the conference room had significantly higher synchrony (H(1) = 7.056, p = 0.0079) than the teams in the garage. This data was surprising, unexpected, and difficult to explain. In the course of searching for an explanation, several earlier models of behavior and context from the literature were reviewed. This led to the development of a comprehensive model of human-environment interaction which we believe will help guide future experiments. Early prototypes of this model are presented and discussed.

The goal of the paper is to introduce a Virtual Reality Model Assessment Platform. VRMAP (for short) is to be a lightweight, simple to use platform that allows users from various fields to easily check out models in virtual reality, as there are few options to do so natively on the headset. The platform consists of a Model Repository and VR client app. Models can be uploaded to the Model Repository and then downloaded and spawned in the client app for assessment or observation. The VR client app is created in Unity Engine using the new Unity XR framework and supports the GL Transmission Format (glTF), a royalty-free, extensible, common publishing format for 3D models created by Khronos Group. Many of the mainstream formats used in 3D printing and computer graphics can be easily converted to glTF and thus imported into the VRMAP. The Model Repository is built using Express, Node, Typescript and MongoDB. It allows for upload, storage, and download of models. Although this implementation is fully functional, it acts as an example and as long as other, more robust implementation adheres to the defined API standard, it will work as a drop-in replacement.

Teaching learning process is seeing a large transformation in this digital age. It involves digital classrooms with various accessories of online tools such as video-conferencing, digital materials, and other platforms for learning and assessment with options for both real-time and self-paced work in addition to the availability of teachers over video-conferencing, text, phone, email, etc. Traditional classrooms are transforming into smart classrooms with the inclusion of the latest technologies and offering an interactive learning environment for students. Online tools such as video-conferencing, multimedia lessons, digital materials, and e-learning platforms with options for both real-time learning and self-paced learning provide a pleasant and immersive experience. In addition to these features, assessment of cognitive state during the learning phase has been proven to improve the efficiency of online learning. Here, we focus toward the analysis of cognitive states over diverse learning assignments using electroencephalogram (EEG) signals that were obtained by means of 128 channels emotive epoch headset device. The artifacts present in original signals were filtered by using linear filtering. So as to determine the exact concentration levels, the fuzzy fractal dimension measures and the discrete wavelet transform were adapted to the same extracted EEG signals for extracting he features. These extracted parameters were then classified into concentration levels using the prominent deep learning methodology, the enhanced convolutional neural network (ECNN), which was proven to be of higher accuracy compared to other classifiers. ECNN can then be used to control cognitive states as a feedback mechanism.

Access to affordable, objective and scalable biomarkers of brain function is needed to transform the healthcare burden of neuropsychiatric and neurodegenerative disease. Electroencephalography (EEG) recordings, both resting and in combination with targeted cognitive tasks, have demonstrated utility in tracking disease state and therapy response in a range of conditions from schizophrenia to Alzheimer's disease. But conventional methods of recording this data involve burdensome clinic visits, and behavioural tasks that are not effective in frequent repeated use. This paper aims to evaluate the technical and human-factors feasibility of gathering large-scale EEG using novel technology in the home environment with healthy adult users. In a large field study, 89 healthy adults aged 40–79 years volunteered to use the system at home for 12 weeks, 5 times/week, for 30 min/session. A 16-channel, dry-sensor, portable wireless headset recorded EEG while users played gamified cognitive and passive tasks through a tablet application, including tests of decision making, executive function and memory. Data was uploaded to cloud servers and remotely monitored via web-based dashboards. Seventy-eight participants completed the study, and high levels of adherence were maintained throughout across all age groups, with mean compliance over the 12-week period of 82% (4.1 sessions per week). Reported ease of use was also high with mean System Usability Scale scores of 78.7. Behavioural response measures (reaction time and accuracy) and EEG components elicited by gamified stimuli (P300, ERN, Pe and changes in power spectral density) were extracted from the data collected in home, across a wide range of ages, including older adult participants. Findings replicated well-known patterns of age-related change and demonstrated the feasibility of using low-burden, large-scale, longitudinal EEG measurement in community-based cohorts. This technology enables clinically relevant data to be recorded outside the lab/clinic, from which metrics underlying cognitive ageing could be extracted, opening the door to potential new ways of developing digital cognitive biomarkers for disorders affecting the brain.

Current Virtual Reality (VR) systems are bereft of stylization and embellishment of the user's motion - concepts that have been well explored in animations for games and movies. We present CooIMoves, a system for expressive and accentuated full-body motion synthesis of a user's virtual avatar in real-time, from the limited input cues afforded by current consumer-grade VR systems, specifically headset and hand positions. We make use of existing motion capture databases as a template motion repository to draw from. We match similar spatio-temporal motions present in the database and then interpolate between them using a weighted distance metric. Joint prediction probability is then used to temporally smooth the synthesized motion, using human motion dynamics as a prior. This allows our system to work well even with very sparse motion databases (e.g., with only 3-5 motions per action). We validate our system with four experiments: a technical evaluation of our quantitative pose reconstruction and three additional user studies to evaluate the motion quality, embodiment and agency., Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 5 (2), ISSN:2474-9567

Robots are expected to become an essential part in tackling the global corona virus pandemic. However, acquiring robotic technology in economically-challenged nations is very difficult for a variety of factors. This work aims at developing a low-cost Telexistence systems that can be used for remote manipulation. We propose a system comprising components on the local site, which comprise a commonly available VR headset, a tracking system and a 3D-printed haptic exoskeleton. The remote site comprises a small robot arm. We discuss the implementation specifics of our approach. We evaluated our system by comparing usability and performance in two tasks while using the VR headset and a monitor. Users were able to complete the two tasks successfully with our system. Moreover, results show superiority of the VR headset, however, the screen also show promising results, and thereby offer a cheaper deployment option.

Background: Several clinical studies have shown that neurofeedback (NFB) has the potential to significantly improve the quality of life of patients complaining of chronic subjective tinnitus. Yet the clinical applicability of such a therapeutic approach in the everyday practice has not been tested so far. Objective: This study aims at investigating the feasibility and efficacy of a semi-automated NFB intervention by means of a portable device that eventually could be used by the patients at home on an everyday basis. The duration of setup procedures is minimized through the use of a dry electrodes electroencephalography (EEG) headset and an automated user-interface. Methods: We conducted a pilot clinical study (non-controlled, single arm, NCT03773926 ). According to a predetermined power calculation, a homogeneous population of 33 subjects with strict inclusion criteria was enrolled. After inclusion, all patients underwent 10 NFB sessions lasting 50 min each, over a period of 5 weeks and a 3-month follow-up period. According to previous studies, the NFB training aimed at increasing the alpha-band power (8–12 Hz) in the EEG power spectrum on the averaged signal of leads FC1, FC2, F3 and F4. Tinnitus handicap inventory (THI) was used as a primary outcome measure. Secondary outcome measures were the visual analog scales (VAS) and the change of the alpha-band power within sessions and across training. Time points of assessment were before intervention (T1), after intervention (T2) and at the 3-month follow-up (T3). Results: Patient exhibited a clinically significant decrease of the THI score, with a 23% decrease (N = 28) on average between T1 and T2 and a 31% decrease (N = 25) between T1 and T3. A significant increase of the alpha-band power within sessions was observed. No significant increase of the alpha-band power across sessions was observed. For the 19 subjects where sufficient data were exploitable, a significant correlation was found between the evolution of the alpha-band training across sessions and the evolution of the THI between T1 and T2. The sessions were well tolerated and no adverse effect was reported. Conclusion: This study suggests that neurofeedback has potential to suit everyday clinical practice with the goal to significantly reduce tinnitus intrusiveness. The merits and limitations of this NFB procedure are discussed, especially with respect to the choice of EEG electrodes to ensure a good signal quality.