Search

Virtual reality (VR) and augmented reality (AR) applications are very diffuse nowadays. Moreover, recent technology innovations led to the diffusion of commercial head-mounted displays for immersive VR: users can enjoy entertainment activities that fill their visual fields, experiencing the sensation of physical presence in these virtual immersive environments. Even if AR and VR are mostly used separately, they can be effectively combined to provide a multi-user shared environment (SE), where two or more users perform some specific tasks in a cooperative or competitive way, providing a wider set of interactions and use cases compared to immersive VR alone. However, due to the differences between the two technologies, it is difficult to develop SEs offering a similar experience for both AR and VR users. This paper presents Harmonize, a novel framework to deploy applications based on SEs with a comparable experience for both AR and VR users. Moreover, the framework is hardware-independent, and it has been designed to be as much extendable to novel hardware as possible. An immersive game has been designed to test and to evaluate the validity of the proposed framework. The assessment of the system through the System Usability Scale questionnaire and the Game Experience Questionnaire shows a positive evaluation.

Brain Computer Interfaces (BCIs) proved to overcome some limitations of other input modes (e.g., gestures, voice, haptic, etc.). BCIs are able to detect the brain activity, thus identifying searched patterns. When a specific brain activity is detected, a well-defined action can be triggered, thus implementing a human-machine interaction paradigm. BCIs can be used in different domains ranging from industry to services for impaired people.Small and ergonomics devices, such as the NextMind (https://www.next-mind.com/) are the result of recent technological advances; these new devices allow to support users in everyday life, thus bringing the design of BCIs into a new dimension well beyond the scope of laboratory tests.In particular, The NextMind is a device able to detect and classify signals coming from the visual cortex. Visual stimuli are blinking/flickering textures that are associated with objects called NeuroTags (see Figure 1). An event is triggered when the user focuses on the same NeuroTag for a given amount of time. This paradigm can replace selection methods based on keyboard, mouse, gesture, touch, voice, and gaze.This paper compares and assesses three different interfaces that share the same input device (the NextMind) to detect the brain activity and differ in the medium to convey to the user the visual stimuli. A monitor, an Augmented Reality (AR) device (the Microsoft HoloLens), and a Virtual Reality (VR) device (the Oculus Rift) are considered. The aim of this work is to assess any difference in the three visualization media when displaying NeuroTags. User tests have been performed in order to evaluate the usability of the three different solutions. After each test, users were asked for filling out the System Usability Scale (SUS) questionnaire and the SUS scores have been used for statistical analysis.

Many studies have focused on Virtual Reality (VR) frameworks for remotely controlling robotic systems. Although VR systems have been used to teleoperate robots in simple scenarios, their effectiveness in terms of accuracy, speed, and usability has not been rigorously evaluated for complex tasks that require accurate trajectories. In this work, an Enhanced Virtual Reality (EVR) framework for robotic teleoperation is evaluated to assess if it can be efficiently used in complex tasks that require accurate control of the robotic end-effector. The environment and the employed robot are captured using RGB-D cameras, while the remote user controls the motion of the robot with VR controllers. The captured data are transmitted and reconstructed in 3D so as to allow the remote user to monitor the task execution progress in real time, using a VR headset. The EVR system is compared with two other interface alternatives: i) teleoperation in pure VR (the model of the robot is rendered with respect to its real joint states), and ii) teleoperation in EVRR (the model of the robot is superimposed on the real robot). The results show that pure point cloud interfaces suffer from visualization issues, reducing the effectiveness of the robot teleoperation. However, the accuracy and user experience can be greatly improved by including the robot model.

In this work, a Mixed Reality (MR) system is evaluated to assess whether it can be efficiently used in teleoperation tasks that require an accurate control of the robot end-effector. The robot and its local environment are captured using multiple RGB-D cameras, and a remote user controls the robot arm motion through Virtual Reality (VR) controllers. The captured data is streamed through the network and reconstructed in 3D, allowing the remote user to monitor the state of execution in real time through a VR headset. We compared our method with two other interfaces: i) teleoperation in pure VR, with the robot model rendered with the real joint states, and ii) teleoperation in MR, with the rendered model of the robot superimposed on the actual point cloud data. Preliminary results indicate that the virtual robot visualization is better than the pure point cloud for accurate teleoperation of a robot arm.

The introduction of interactive assets in public exhibitions is capable to significantly enhance the visitors’ user experience. However, the creation of interactive applications could represent a challenging task, especially for users lacking computer skills. Visual programming languages (VPLs) – one of the instruments belonging to the broad categories of methods and tools devised to support end-user development (EUD) – promise to offer an intuitive way to overcome these limitations, by providing easy-to-use and efficient interfaces for encoding applications’ logic. Moving from these considerations, this paper first analyses pros and cons of tools devised so far to support the generation of interactive contents. Then, it presents the design of a new tool named Visual Scene Editor (VSE), which allows users with little to no programming skills to create 3D interactive applications by combining available assets through an interactive, visual process. Both objective and subjective measurements have been collected with both skilled and unskilled users to evaluate the performance of the proposed tool. A comparison with existing solutions shows a reduction in the time required to complete the assigned tasks, of the complexity of the logic created, as well as of the number of errors made, confirming the suitability of the VSE for the said purpose.

Until some years ago, there was a lack of affordable devices that allowed for research in order to design and implement augmented and mixed reality applications. Thanks to technology improvements, it is now possible to find several wearable devices on the market that greatly improve the synergy between the real and virtual world, thus allowing users to be part of an immersive experience in which they can easily interact with 3D assets and real objects at the same time. Thus, the number of augmented reality and mixed reality (AR/MR) applications is greatly increased, from educational or cultural heritage applications to industry ones, such as maintenance-assembly-repair processes or product inspection and building monitoring. Indeed, several improvements have been made in order to enhance this technology, but some issues still need to be tackled, namely: limited field-of-view, tracking stability, the effectiveness of user interfaces to interact with 3D contents, and many others.

Augmented Reality (AR) and Virtual Reality (VR) have been usually addressed as two separated worlds and recent studies try to address the problem of merging the AR and VR applications into a single “environment”, providing a system that relies on both paradigms. The constant release of new hardware interfaces for both wearable AR and Immersive VR opens up new possibilities for the gaming area and many others. However, even if there are researches that explore the usage of AR and VR in the same application, videogames are deployed for one environment or the other depending on their strengths and flaws and the type of experience they can offer to the player, in order to exalt the peculiarities of the chosen medium. A novel approach would be to provide a multiplayer system that enables the users to play the same (or similar) experience through either an AR or VR interface: the player could freely choose the interface, based on several factors such as hardware availability, environment, physical limitations or personal preferences. In this paper, a preliminary study on a multiplayer game system for both AR and VR interfaces is proposed. A chess game experience is provided and a comparison through a System Usability Scale (SUS) questionnaire allowed to establish if both interfaces provided a satisfactory game experience and to highlight both hardware limitations and further interface enhancements.

Augmented Reality (AR) and Virtual Reality (VR) have been usually addressed as two separated worlds and recent studies try to address the problem of merging the AR and VR applications into a single “environment”, providing a system that relies on both paradigms. The constant release of new hardware interfaces for both wearable AR and Immersive VR opens up new possibilities for the gaming area and many others. However, even if there are researches that explore the usage of AR and VR in the same application, videogames are deployed for one environment or the other depending on their strengths and flaws and the type of experience they can offer to the player, in order to exalt the peculiarities of the chosen medium. A novel approach would be to provide a multiplayer system that enables the users to play the same (or similar) experience through either an AR or VR interface: the player could freely choose the interface, based on several factors such as hardware availability, environment, physical limitations or personal preferences. In this paper, a preliminary study on a multiplayer game system for both AR and VR interfaces is proposed. A chess game experience is provided and a comparison through a System Usability Scale (SUS) questionnaire allowed to establish if both interfaces provided a satisfactory game experience and to highlight both hardware limitations and further interface enhancements.

Industry 4.0 is moving factories towards a high level of collaboration between human workers and industrial robots, with the aim of improving efficiency and productivity. Among other technologies, Augmented Reality (AR) is one of the most researched in recent years to provide novel user interfaces that could easily blend the real world with additional information. This literature review aims at identifying the main strengths and weaknesses of AR with industrial robots in human–robot collaborative scenarios. The term industrial robot is meant according to the ISO 8373:2012 definition. To this end, starting from 3734 initial works, 63 papers have been selected and analysed. The results suggest that AR technology shows its effectiveness also in this particular domain. With respect to traditional approaches, AR systems are faster and more appreciated by users. Nonetheless, the use of AR in human–robot collaborative scenarios is so cutting edge that not all the considered works have properly evaluated the proposed user interfaces. Future research should improve the qualitative evaluation in order to clearly point out both limitations and strengths of the proposed systems, involving also expert users in tests.

Technology advancement is changing the way industrial factories have to face an increasingly complex and competitive market. The fourth industrial revolution (known as industry 4.0) is also changing how human workers have to carry out tasks and actions. In fact, it is no longer impossible to think of a scenario in which human operators and industrial robots work side-by-side, sharing the same environment and tools. To realize a safe work environment, workers should trust robots as well as they trust human operators. Such goal is indeed complex to achieve, especially when workers are under stress conditions, such as when a fault occurs and the human operators are no longer able to understand what is happening in the industrial manipulator. Indeed, Augmented Reality (AR) can help workers to visualize in real-time robots’ faults. This paper proposes an augmented system that assists human workers to recognize and visualize errors, improving their awareness of the system. The system has been tested using both an AR see-through device and a smartphone.

Since the origins of Augmented Reality (AR), industry has always been one of its prominent application domains. The recent advances in both portable and wearable AR devices and the new challenges introduced by the fourth industrial revolution (renowned as industry 4.0) further enlarge the applicability of AR to improve the productiveness and to enhance the user experience. This paper provides an overview on the most important applications of AR regarding the industry domain. Key among the issues raised in this paper are the various applications of AR that enhance the user's ability to understand the movement of mobile robot, the movements of a robot arm and the forces applied by a robot. It is recommended that, in view of the rising need for both users and data privacy, technologies which compose basis for Industry 4.0 will need to change their own way of working to embrace data privacy.