Search

The internet-of-things (IoT) refers to the growing field of interconnected pervasive computing devices and the networking that supports smart, embedded applications. The IoT has multiple human-computer interaction challenges due to its many formats and interlinked components, and central to these is the need to provide sensory information and situational context pertaining to users in a more human-friendly, easily understandable format. This work addresses this by applying mixed reality toward expressing the underlying behaviors and states internal to IoT devices and IoT-enabled objects. It extends the authors’ previous research on IoT Avatars (mixed reality character representations of physical IoT devices), presenting a new head-mounted display framework and interconnection architecture. This contributes i) an exploration of mixed reality for smart spaces, ii) an approach toward expressive avatar behaviors using fuzzy inference, and iii) an early functional prototype of a hybrid physical and mixed reality IoT-enabled object. This approach is a step toward new information presentation, interaction, and engagement capabilities for smart devices and environments.

During crisis situations, emergency operators require fast information access to achieve situation awareness and make the best possible decisions. Augmented reality could be used to visualize the wealth of user-generated content available on social media and enable context-adaptive functions for emergency operators. Although emergency operators agree that social media analytics will be important for their future work, it poses a challenge to filter and visualize large amounts of social media data. We conducted a goal-directed task analysis to identify the situation awareness requirements of emergency operators. By collecting tweets during two storms in Germany we evaluated the usefulness of Twitter data for achieving situation awareness and conducted interviews with emergency operators to derive filter strategies for social media data. We synthesized the results by discussing how the unique interface of augmented reality can be used to integrate social media data into emergency control rooms for situation awareness support.

Background This review aims to examine the existing literature to address currently used virtual, augmented, and mixed reality modalities in the areas of preoperative surgical planning, intraoperative guidance, and postoperative management in the field of cardiothoracic surgery. In addition, this innovative technology provides future perspectives and potential benefits for cardiothoracic surgeons, trainees, and patients. Methods A targeted, non-systematic literature assessment was performed within the Medline and Google Scholar databases to help identify current trends and to provide better understanding of the current state-of-the-art extended reality (XR) modalities in cardiothoracic surgery. Related articles published up to July 2020, are included in the review. Results XR is a novel technique gaining increasing application in cardiothoracic surgery. It provides a three-dimensional (3D) and realistic view of structures and environments and offers the user the ability to interact with digital projections of surgical targets. Recent studies showed the validity and benefits of XR applications in cardiothoracic surgery. Examples include XR-guided pre-operative planning, intraoperative guidance and navigation, post-operative pain and rehabilitation management, surgical simulation, and patient education. Conclusions XR is gaining interest in the field of cardiothoracic surgery. In particular, there are promising roles for XR applications in televirtuality, surgical planning, surgical simulation, and perioperative management. However, future refinement and research is needed to further implement XR in the aforementioned settings within cardiothoracic surgery.

OBJECTIVES Extended reality (XR), encompassing both virtual reality (VR) and augmented reality, allows the user to interact with a computer-generated environment based on reality. In essence, the immersive nature of VR and augmented reality technology has been warmly welcomed in all aspects of medicine, gradually becoming increasingly feasible to incorporate into everyday practice. In recent years, XR has become increasingly adopted in thoracic surgery, although the extent of its applications is unclear. Here, we aim to review the current applications of XR in thoracic surgery. METHODS A systematic database search was conducted of original articles that explored the use of VR and/or augmented reality in thoracic surgery in EMBASE, MEDLINE, Cochrane database and Google Scholar, from inception to December 2020. RESULTS Our search yielded 1494 citations, of which 21 studies published from 2007 to 2019 were included in this review. Three main areas were identified: (i) the application of XR in thoracic surgery training; (ii) preoperative planning of thoracic procedures; and (iii) intraoperative assistance. Overall, XR could produce progression along the learning curve, enabling trainees to reach acceptable standards before performing in the operating theatre. Preoperatively, through the generation of 3D-renderings of the thoracic cavity and lung anatomy, VR increases procedural accuracy and surgical confidence through familiarization of the patient’s anatomy. XR-assisted surgery may have therapeutic use particularly for complex cases, where conventional methods would yield inadequate outcomes due to inferior accuracy. CONCLUSION XR represents a salient step towards improving thoracic surgical training, as well as enhancing preoperative planning and intraoperative guidance., Extended reality (XR), encompassing both virtual (VR) and augmented reality (AR), allows the user to interact with a computer-generated environment based on reality.

We present the case of a 74-year-old man with a history of a squamous cell carcinoma in the left axilla. The patient underwent a multidisciplinary surgical resection through an extended forequarter amputation with thoracic wall resection and reconstruction. With regard to the complexity of the case, three-dimensional virtual reality–based patient-specific reconstructions were used as a supplemental tool to conventional computed tomography imaging to plan the procedure. With this report, we aim to stimulate further research to improve and automate the workflow and to bring virtual and augmented reality reconstructions into the surgical theater of the future.

Mutual occlusion is an essential feature for augmented reality (AR) displays for allowing the virtual content to be clearly perceived under an excessively illuminated environment. Although a few works have been done to facilitate the performance of occlusion-capable optical see-through augmented reality (OC-OST-AR) displays, the realization of mutual occlusion in a wide field-of-view (FOV) is still challenging. Divergent from typical hard-edge occlusion and soft edge-occlusion designs, we propose the paired-ellipsoidal-mirror (PEM) structure. The proposed system is allowed to support either hard-edge occlusion or enhanced soft-edge occlusion in a wide FOV by optionally fixing a spatial light modulator (SLM) before the entrance pupil or at an inner focal plane. The numerical aperture (NA) of the system is efficiently increased by the combination of paired ellipsoidal mirror imaging and aperture stop restriction. With proof-of-concept prototypes built, virtual display in a FOV of H160°×V74° and mutual occlusion in a FOV of H122°×V74° are demonstrated with a basic design, respectively. Furthermore, a mixed FOV of H95.3°×V52.9° is demonstrated by an optimized design with vertical parallax reduction and virtual display improvement.

The amount of arable land is limited, yet the demand for agricultural food products is increasing. This issue has led to the notion of precision farming, where smart city-based technologies (e.g., Internet of Things, digital twins, artificial intelligence) are employed in combination to cater for increased production with fewer resources. Widely used in manufacturing, augmented reality has demonstrated impactful solutions for information communication, remote monitoring and increased interaction. Yet, the technology has only recently begun to find a footing alongside precision farming solutions, despite the many benefits possible to farmers through augmenting the physical world with digital objects. Therefore, this article reflects on literature discussing current applied solutions within agriculture, where augmented realty has demonstrated a significant impact for monitoring and production. The findings discuss that augmented reality must be coupled with other technologies (e.g., simultaneous localization and mapping algorithms, global positioning systems, and sensors), specifically 9 are identified across 2 application domains (livestock and crop farming) to be beneficial. Attention is also provided on how augmented reality should be employed within agriculture, where related-work examples are drawn from in order to discuss suitable hardware approaches and constraints (e.g., mobility).

Cultural-heritage research has created a vast amount of information regarding heritage objects and sites. At the same time, recent efforts on the digitization of cultural heritage have provided novel means of documenting tangible cultural resources including digital images, videos, audio testimonies, and 3D reconstructions. Furthermore, ethnographic research is nowadays combined with advanced capturing technologies such as motion capture to record the intangible dimensions of heritage as these are manifested through human expression in dance, heritage crafts, etc. This amount of information is now available and should be used to create novel forms of experiential access to cultural heritage powered by the web and mobile technologies mixed with novel interaction paradigms such as virtual and augmented reality. In this article, a platform is presented that facilitates a cloud-based web application to support location-based narratives on cultural-heritage resources provided through map-based or story-based representation approaches. At the same time, the platform through the power of modern mobile devices can provide these experiences on the move using location-based and image recognition-based augmented reality to facilitate multiple usage contexts. The platform was implemented to support the goal of the project CuRe, in the context of the “Greece-Germany” bilateral collaboration action.

The rapid development of virtual and augmented reality technologies is currently taking place in almost all spheres of activity. Elements of virtual and augmented reality are used in such areas as education, medicine, transport, gaming, tourism and others. The active spread of these technologies causes the emergence of special competencies in the IT labor market and, as a result, the formation of new professions.Many Russian universities are training students in IT training areas. Specialization in the development of computer games and virtual reality applications has begun recently. The provision of practical classes is accompanied by specific tasks, which gives students the opportunity to improve the use of software and technical devices.The relevance of the research is determined by the current demand for the use of the latest technologies by IT developers in the field of creating computer games. Today, technologies that provide a player’s immersion in virtual reality are becoming more and more popular. One of these technologies is a suit with wearable sensors that track a person’s position in space in real time. However, there are quite a few real described projects in the literature and on the Internet. This study examines the process of developing a task for creating a game application using virtual reality technology: a suit with wearable sensors for teaching students.Materials and methods of research. Timely identification of the needs of the IT market in personnel training allows educational organizations to form new training programs of different levels of training. This approach makes it possible to target the educational and methodological materials being developed to use the latest achievements in the development of the field under study.Using a systematic approach, the study characterizes virtual reality suits and sensors for monitoring the position in the user’s space. Thus, the goal of the task was to ensure the immersiveness and convenience of interaction between the player and the game environment.Based on materials on software, position sensors in space, the approach of pedagogical design was applied and the procedure was formed for a practical task, reflecting the relevant competencies.Results. The study was conducted on the basis in the framework of laboratory and practical work of students, as well as at a real enterprise. Training in the new profile of the direction of training “Applied informatics” is fully equipped with all the latest technologies in this field. As a result of the work, the content of the practical task was developed.Real development of virtual and augmented reality applications is conducted jointly with students. Almost all projects used a suit with body sensors.Conclusion. Our study examines in detail the process of developing an application using a suit with wearable sensors for further training of students. Based on the results, work can be carried out on real projects for any field. Based on the research materials, it is planned to issue a textbook for students with the profile of developing computer games and virtual / augmented reality applications.

To help individuals deal with volleyball learning and training, an online volleyball remote teaching system is designed based on Augmented Reality (AR) technology. Firstly, the action images of the volleyball coach were captured, followed by being input into the computer. The hardware of the system was mainly composed of motion capture equipment, data transmission equipment, central control processing equipment, and AR display equipment. The system’s software included four modules of motion acquisition and recognition. The testing results showed that the functional modules and performance of the system could meet the expected requirements, verifying the effectiveness and practicability.

In the modern era, preoperative planning is substantially facilitated by artificial reality technologies, which permit a better understanding of patient anatomy, thus increasing the safety and accuracy of surgical interventions. In the field of orthopedic surgery, the increase in safety and accuracy improves treatment quality and orthopedic patient outcomes. Artificial reality technologies, which include virtual reality (VR), augmented reality (AR), and mixed reality (MR), use digital images obtained from computed tomography or magnetic resonance imaging. VR replaces the user's physical environment with one that is computer generated. AR and MR have been defined as technologies that permit the fusing of the physical with the virtual environment, enabling the user to interact with both physical and virtual objects. MR has been defined as a technology that, in contrast to AR, enables users to visualize the depth and perspective of the virtual models. We aimed to shed light on the role that MR can play in the visualization of orthopedic surgical anatomy. The literature suggests that MR could be a valuable tool in orthopedic surgeon's hands for visualization of the anatomy. However, we remark that confusion exists in the literature concerning the characteristics of MR. Thus, a more clear description of MR is needed in orthopedic research, so that the potential of this technology can be more deeply understood.

BACKGROUND CONTEXT: The field of artificial intelligence (AI) is rapidly advancing, especially with recent improvements in deep learning (DL) techniques. Augmented (AR) and virtual reality (VR) are finding their place in healthcare, and spine surgery is no exception. The unique capabilities and advantages of AR and VR devices include their low cost, flexible integration with other technologies, user-friendly features and their application in navigation systems, which makes them beneficial across different aspects of spine surgery. Despite the use of AR for pedicle screw placement, targeted cervical foraminotomy, bone biopsy, osteotomy planning, and percutaneous intervention, the current applications of AR and VR in spine surgery remain limited. PURPOSE: The primary goal of this study was to provide the spine surgeons and clinical researchers with the general information about the current applications, future potentials, and accessibility of AR and VR systems in spine surgery. STUDY DESIGN/SETTING: We reviewed titles of more than 250 journal papers from google scholar and PubMed with search words: augmented reality, virtual reality, spine surgery, and orthopaedic, out of which 89 related papers were selected for abstract review. Finally, full text of 67 papers were analyzed and reviewed. METHODS: The papers were divided into four groups: technological papers, applications in surgery, applications in spine education and training, and general application in orthopaedic. A team of two reviewers performed paper reviews and a thorough web search to ensure the most updated state of the art in each of four group is captured in the review. RESULTS: In this review we discuss the current state of the art in AR and VR hardware, their preoperative applications and surgical applications in spine surgery. Finally, we discuss the future potentials of AR and VR and their integration with AI, robotic surgery, gaming, and wearables. CONCLUSIONS: AR and VR are promising technologies that will soon become part of standard of care in spine surgery. (C) 2021 Published by Elsevier Inc.

Marc-doa application is Mobile Augmented Reality cubes used as a learning media for children, focusing on cognitive and psychomotor learning. The contents of this application are daily prayers for Muslim children and designed in the form of audio-video animations. The development of the Marc-doa application uses the Multimedia Development Life Cycle method. This research aims to discuss the design and evaluation of Marc-doa. Evaluation and technical tests are conducted on several aspects, specifically on the detection process, distance, angle, and surface area. In addition, the testing process is to identify the ideal conditions for the use of Marc-doa. From the evaluation and testing process, four markers showed good results. Only one marker does not show the results as expected. The ideal distance for using Marc-doa application between 25-40 cm. The angle for ideal reception between 0-30 degrees and the surface area Marc-doa application can detect markers correctly between 50%-100% surface area. The ideal distance, surface area, and conditions for utilising Marc-doa application were obtained from this research.

A museum can be a suitable context to experiment with a new interaction technique which guiding visitors and improving their experience for a successful learning environment also to enrich user experience. Mobile technology is used to enhance user experience in a cultural environment that offers more information than physically exposed technology. Therefore, the introduction of augmented reality (AR) that allow visitor to experience interaction in a way that increased knowledge, learning and give additional information with any artefact exhibitions they interact in the museum is applied in this project. The objectives of this research are to develop an interactive design in enhancing user experience in artefact display and evaluate user experience. ADDIE Model is applied as the methodology that consists of five phases which are analysis, design, development, implementation, and evaluation. Principles of Interactive Design is utilized in designing the mobile AR application and developed using Unity, Blender and Vuforia. User Experience evaluation method is conducted where findings shows that most respondents are satisfied, and usage of mobile AR has enhanced their experience through navigation during museum visit that also give a lot of new insight on historical artefact that being displayed.

The design of user interfaces (UIs) for assistive robot systems can be improved through the use of a set of design guidelines presented in this article. As an example, the article presents two different UI designs for an assistive manipulation robot system. We explore the design considerations from these two contrasting UIs. The first is referred to as the graphical user interface (GUI), which the user operates entirely through a touchscreen as a representation of the state of the art. The second is a type of novel UI referred to as the tangible user interface (TUI). The TUI makes use of devices in the real world, such as laser pointers and a projector–camera system that enables augmented reality. Each of these interfaces is designed to allow the system to be operated by an untrained user in an open environment such as a grocery store. Our goal is for these guidelines to aid researchers in the design of human–robot interaction for assistive robot systems, particularly when designing multiple interaction methods for direct comparison.

Public bus transportation has become an integral part of society, but the disrup-tion of bus services is one of the major concerns. This project presents the devel-opment of Smart Bus Transportation using Augmented Reality (TRANSPAR) that was developed on a mobile application. One of the major issues with public transportation is on real-time responsiveness. Most bus schedules are presented online but customers still faced many failures. Some bus schedules are not updat-ed when changes happened through time. Some existing bus schedules system is fixed to the bus stations. This research is to identify the bus schedules and its routes characteristics. A 3D AR animation based on identified characteristics was designed using the Unity 3D image marker detection on a mobile Android plat-form. A smartphone application was developed using Vuforia and Google Fire-base. TRANSPAR shows an AR mobile application for acquiring the bus time-tables. The phone camera is applied for marker image detection and scanning the bus station’s images. AR and normal image scanner were designed. Google Fire-base Database is used to retrieve and store each timetable data for every bus sta-tion. Analysis of interactivity and benefits of TRANSPAR shows about 90% agreed on the use of AR and more than 76% agreed on its functionality based on 50 taken samples. This shows a positive impact on the designed TRANSPAR. The research is significant to encourage and experience the public with new tech-nological application for public transportation and it impact the society.

Una de las ramas de la arquitectura que más se ha visto impactada por la digitalización es la representación, entendida esta como tarea esencial en el proceso de diseño, creación y comunicación del proyecto arquitectónico, realizada por medio de herramientas digitales que progresivamente han venido ganando terreno en el quehacer técnico y profesional de la construcción. El presente trabajo analítico descriptivo, de enfoque cuantitativo, se basó en analizar las técnicas digitales aplicadas para la representación de los proyectos arquitectónicos en Nuevo León, México. Su importancia radica en lo relevante que resulta la actividad de la industria del diseño y la construcción, para la económica regional y nacional. Entre los resultados del estudio resalta el uso privilegiado de los dibujos digitales en 2D, por parte de los arquitectos informantes, ya que estos son considerados como expresiones originarias del diseño arquitectónico; en igual medida le siguen los dibujos digitales en 3D de representación formal y volumétrica. Cómo técnicas complementarias se encuentran el uso y edición digital de imágenes y videos, así como también las técnicas de fabricación digital. Mientras que la realidad virtual, y la realidad aumentada, no se consideraron como parte esencial de la representación digital del proyecto arquitectónico.

BackgroundOral and maxillofacial surgery currently relies on virtual surgery planning based on image data (CT, MM). Three-dimensional (3D) visualizations are typically used to plan and predict the outcome of complex surgical procedures. To translate the virtual surgical plan to the operating room, it is either converted into physical 3D-printed guides or directly translated using real-time navigation systems.PurposeThis study aims to improve the translation of the virtual surgery plan to a surgical procedure, such as oncologic or trauma surgery, in terms of accuracy and speed. Here we report an augmented reality visualization technique for image-guided surgery. It describes how surgeons can visualize and interact with the virtual surgery plan and navigation data while in the operating room. The user friendliness and usability is objectified by a formal user study that compared our augmented reality assisted technique to the gold standard setup of a perioperative navigation system (Brainlab). Moreover, accuracy of typical navigation tasks as reaching landmarks and following trajectories is compared.ResultsOverall completion time of navigation tasks was 1.71 times faster using augmented reality (P = .034). Accuracy improved significantly using augmented reality (P < .001), for reaching physical landmarks a less strong correlation was found (P = .087). Although the participants were relatively unfamiliar with VR/AR (rated 2.25/5) and gesture-based interaction (rated 2/5), they reported that navigation tasks become easier to perform using augmented reality (difficulty Brainlab rated 3.25/5, HoloLens 2.4/5).ConclusionThe proposed workflow can be used in a wide range of image-guided surgery procedures as an addition to existing verified image guidance systems. Results of this user study imply that our technique enables typical navigation tasks to be performed faster and more accurately compared to the current gold standard. In addition, qualitative feedback on our augmented reality assisted technique was more positive compared to the standard setup. (C) 2021 The Author. Published by Elsevier Inc. on behalf of The American Association of Oral and Maxillofacial Surgeons.

Traditional neurosurgical craniotomy primarily uses two‐dimensional cranial medical images to estimate the location of a patient’s intracranial lesions. Such work relies on the experience and skills of the doctor and may result in accidental injury to important intracranial physiological tissues. To help doctors more intuitively determine patient lesion information and improve the accuracy of surgical route formulation and craniotomy safety, an augmented reality method for displaying neurosurgery craniotomy lesions based on feature contour matching is proposed. This method uses threshold segmentation and region growing algorithms to reconstruct a 3‐D Computed tomography image of the patient’s head. The augmented reality engine is used to adjust the reconstruction model’s relevant parameters to meet the doctor’s requirements and determine the augmented reality matching method for feature contour matching. By using the mobile terminal to align the real skull model, the virtual lesion model is displayed. Using the designed user interface, doctors can view the patient’s personal information and can zoom in, zoom out, and rotate the virtual model. Therefore, the patient’s lesions information can be visualized accurately, which provides a visual basis for preoperative preparation.

The use of Augmented Reality (AR) technology is widespread in countless archaeological sites and a variety of applications. Archaeological excavations lead to archaeological finds, some of which are transported for preservation and then for exhibition in museums (jewelry, vases, etc.), while another part of them is documented in detail and remains in situ (eg building walls), roads, grave covers, etc.). However, after the registration of the archaeological finds, it is impossible to observe them. As part of our research project, we will develop for the first time AR methodology and procedures for the observation of covered archaeological finds on mobile devices (smart phones, tablets), which were registered after their documentation. AR technology in recent years has seen great growth in terms of implementation platforms and available software, as well as the tools developed to support it. These tools either make their appearance in the form of frameworks, extending the capabilities of an existing engine, or function as independent services. At the same time, progress has been made in the field of sensors of mobile devices, which makes the compatibility of hardware and software another issue to be researched. As part of the development of the above application for mobile devices, an evaluation is made of the most widespread AR Frameworks that support the Unity3d Game Engine and the compatibility / interoperability with the sensors of different categories of mobile devices. The frameworks were checked and evaluated for placement and tracking of the positions of the 3D covered objects. In this paper also, methodologies and techniques used in space detection and tracking are presented and evaluated.

Purpose A systematic review of the literature has been carried out to assess the actual evidence of the use of augmented reality in total knee arthroplasty (TKA). We then conducted a pilot clinical study to examine the accuracy of the Knee + augmented reality navigation in performing TKA. The present augmented reality (AR) system allows the surgeon to view the tibial and femur axis superimposed on the surgical field through the smart glasses. It provides real-time information during surgery and intraoperative feedback. Methods A systematic review of the PubMed, MEDLINE, and Embase databases up to May 2021 using the keywords “augmented reality”, “knee arthroplasty”, “computer assisted surgery”, “navigation knee arthroplasty” was performed by two independent reviewers. We performed five TKAs using the Knee + system. Patients were 4 females, with mean age of 76.4 years old (range 73–79) and mean Body Max Index (BMI) of 31.9 kg/m2 (range 27–35). The axial alignment of the limb and the orientation of the components were evaluated on standardized pre and postoperative full leg length weight-bearing radiographs, anteroposterior radiographs, and lateral radiographs of the knee. The time of tourniquet was recorded. The perception of motion sickness was assessed by Virtual Reality Sickness Questionnaire (VRSQ) subjected to surgeon immediately after surgery. Results After duplicate removal, a total of 31 abstracts were found. However, only two studies concerned knee arthroplasty. Unfortunately, both were preclinical studies. Knee + system is able to perform a cutting error of less than 1° of difference about coronal alignment of femur and tibia and less than 2° about flexion/extension of femur and posterior tibial slope. The absolute differences between the values obtained during surgery and the measurement of varus femur, varus tibia, posterior slope, and femur flexion angle on post-operative radiographs were 0.6° ± 1.34°, 0.8° ± 0.84°, 0.8° ± 1.79°, and 0.4 mm ± 0.55 mm, respectively. Conclusions On light of our preliminary results, the Knee + system is accurate and effective to perform TKA. The translation from pilot study to high-level prospective studies is warranted to assess accuracy and cost-effective analysis compared to conventional techniques. Level of evidence IV

The purpose of this paper is to explore how the use of virtual tours may represent a lever to improve customers’ intention to buy food products and/or visit places where products are manufactured. Emerging virtual reality and augmented reality can allow people to interact with products and places where products are produced even from very distant places, with a positive impact on their purchase intentions. Applying the theory of planned behaviour, this study focuses on the Parmigiano Reggiano cheese, one of the most exported Italian foods. A sample of consumers was surveyed online to explore the role of virtually visiting the food production place on consumers’ intention to physically visit the production place and on their intention to buy the food. The empirical results deriving from a structural equation model developed on 399 completed and valid questionnaires indicate that the opportunity to experience products through virtual tours positively influences consumer behaviours. Young males show the highest intention to physically visit the food production place and to buy the cheese. Findings provide new insights for the emerging literature on the use of virtual reality to experience products through virtual interaction and the impact that interactive technologies have on consumer purchase intentions. Local areas can use virtual tours to improve food tourism and stimulate food product sales. To our knowledge, this paper represents one of the first empirical endeavours aimed at exploring the impact of virtual tours on consumers’ buying and visiting intentions when a food product and a dairy are concerned.

Purpose of Review This review summarizes and provides a comprehensive narrative synthesis of the current evidence on immersive technology’s (i.e., virtual and augmented Reality) use for perioperative anxiety, acute, and chronic pain in pediatrics. Recent Findings Researchers have increasingly studied immersive technology as a non-pharmacological alternative for perioperative anxiety, acute, and chronic pain management. We found several research studies published over the last 3 years: almost all studies examined the use of virtual reality for perioperative anxiety and pain; only one case report was about the use of augmented reality for preoperative anxiety. Most studies showed that virtual reality intervention is effective and safe for perioperative anxiety, acute, and chronic pain. However, the studies are heterogeneous with relatively small sample sizes. Summary This review shows that more high-quality studies (i.e., randomized controlled trials with larger sample sizes and standardized methods for measuring and reporting outcomes) are needed to examine the effectiveness and adverse effects of virtual reality intervention on perioperative anxiety, acute, and chronic pain in pediatrics.

The article has dealt with the current trends of communication aspects of company’s relations with the customers. The emphasis is placed on the necessity to review the companies’ marketing strategies, in particular, due to communication constraints and the corresponding changes in the format of both direct and feedback with the customers. Restrictive measures have changed the format of competition, having created the conditions for its concentration in cyberspace. Changes in the nature of interpersonal communication predetermined by the limited personal activity and the introduction of remote forms of interaction, have generated demand for innovative ways of communication with customers. Other factors that require the adjustment of marketing strategies include the emergence of new behavioral capabilities of consumers associated with the intensive development of digital communication channels. It has been noted that in the digital communication environment, digital platforms provide the main format of communication with the existing customer base and potential customers. In addition, prompt acquisition and processing of data on purchasing behavior of customers allows companies to increase significantly the “sensitivity” of marketing research and optimize their own marketing decisions. Among the areas of increasing targeting and personalization of product offerings, the use of elements of artificial intelligence and voice search has been accentuated. Moreover, the use of artificial intelligence greatly facilitates and simplifies the work of marketers on the processing and initial analysis of the growing flow of digital marketing data, as well as provides greater efficiency. It has been concluded that widespread use of voice assistants in e-commerce requires companies to rethink radically their approaches to internal data processing, IT investment and customer engagement strategies. Active integration of chatbots into various communication platforms is able to make them universal tools for both business and consumers, which will allow them to become a familiar part of the user experience. Implementation of virtual reality and augmented reality elements in the interface of company websites can significantly diversify the process of collecting information by users and push them to unplanned purchases. It has been indicated that the growing standards of personalization and new rules of customer confidentiality will continue to require companies to be flexible and introduce new technologies in order to adapt permanently their business to new conditions. Focusing on building trust with customers based on digital communications will enable companies to create a long-term integrated marketing strategy and concentrate on the multi-channel nature of customer experience.

Background Virtual reality (VR) allows for presurgical planning. Intraoperatively, augmented reality (AR) enables integration of segmented anatomic information with neuronavigation into the microsurgical scene to provide guidance without workflow disruption. Combining VR and AR solutions may help guide microsurgical technique to improve safety, efficiency, and ergonomics. Objective To describe a VR/AR platform that provides VR planning and intraoperative guidance via microscope ocular injection of a comprehensive AR overlay of patient-specific 360°/3D anatomic model aligned and synchronized with neuronavigation. Methods Custom 360° models from preoperative imaging of 49 patients were utilized for preoperative planning using a VR-based surgical rehearsal platform. Each model was imported to SyncAR, the platform's intraoperative counterpart, which was coregistered with Medtronic StealthStation S8 and Zeiss or Leica microscope. The model was injected into the microscope oculars and referenced throughout by adjusting overlay opacity. For anatomic shifts or misalignment, the overlay was reregistered via manual realignment with known landmarks. Results No SyncAR-related complications occurred. SyncAR contributed positively to the 3D understanding of patient-specific anatomy and ability to operate. Preoperative planning and intraoperative AR with 360° models allowed for more precise craniotomy planning and execution. SyncAR was useful for guiding dissection, identifying critical structures including hidden anatomy, understanding regional anatomy, and facilitating resection. Manual realignment was performed in 48/49 surgeries. Gross total resection was achieved in 34/40 surgeries. All aneurysm clipping and microvascular decompression procedures were completed without complications. Conclusion SyncAR combined with VR planning has potential to enhance surgical performance by providing critical information in a user-friendly, continuously available, heads-up display format.

With this paper, we present a novel approach for efficiently creating reality-based, high-fidelity urban 3D models for interactive VR cycling simulations. The foundation of these 3D models is accurately georeferenced street-level imagery, which can be captured using vehicle-based or portable mapping platforms. Depending on the desired type of urban model, the street-level imagery is either used for semi-automatically texturing an existing city model or for automatically creating textured 3D meshes from multi-view reconstructions using commercial off-the-shelf software. The resulting textured urban 3D model is then integrated with a real-time traffic simulation solution to create a VR framework based on the Unity game engine. Subsequently, the resulting urban scenes and different planning scenarios can be explored on a physical cycling simulator using a VR helmet or viewed as a 360-degree or conventional video. In addition, the VR environment can be used for augmented reality applications, e.g., mobile augmented reality maps. We apply this framework to a case study in the city of Berne to illustrate design variants of new cycling infrastructure at a major traffic junction to collect feedback from practitioners about the potential for practical applications in planning processes.

Having a good estimate of the position and orientation of a mobile agent is essential for many application domains such as robotics, autonomous driving, and virtual and augmented reality. In particular, when using LiDAR and IMU sensors as the inputs, most existing methods still use classical filter-based fusion methods to achieve this task. In this work, we propose DeepLIO, a modular, end-to-end learning-based fusion framework for odometry estimation using LiDAR and IMU sensors. For this task, our network learns an appropriate fusion function by considering different modalities of its input latent feature vectors. We also formulate a loss function, where we combine both global and local pose information over an input sequence to improve the accuracy of the network predictions. Furthermore, we design three sub-networks with different modules and architectures derived from DeepLIO to analyze the effect of each sensory input on the task of odometry estimation. Experiments on the benchmark dataset demonstrate that DeepLIO outperforms existing learning-based and model-based methods regarding orientation estimation and shows a marginal position accuracy difference.

As the development of information technology and change in people's reading habits, the publishing industry is integrating with the modern information technology and transforming to the digitization. The virtual reality (VR) and augmented reality (AR) technologies have been developed rapidly since 2016, which have begun to be applied in some areas and made remarkable achievements in just a few years. Multiple publishing units in China respond to the call of the times, and conduct the innovative development by combining with their own resource advantages. In this paper, the application status of VR/AR technology in book publishing, newspaper media, periodicals and magazines is summarized, and some representative cases are illustrated the application of VR/AR technologies in the publishing industry does give readers an extraordinary reading experience, and promotes the interaction among readers, authors and publishing units, which finds a new innovation point for the innovation of the whole publishing industry.

Augmented reality (AR) is one of the emerging technologies gradually venturing into the education field. Although AR is strongly linked to subjects related to Science, Mathematics, and Technology in schools and tertiary education, there is no mention of AR in non-technical subjects, such as language. Thus, this study aims to discover the AR application trends in language learning and the language skills prevalent in AR usage. In this systematic literature review, AR-related research in language learning began in 2016 and has continued to be on the rise. Furthermore, the preferred language skills used with AR technology required lower cognitive levels, such as identifying words, understanding meanings, and spelling, and pronouncing words. Based on the gap indicating the rare usage of AR for more complex and critical language skills, such as reading and writing, this study hopes to enlighten the researchers, educators, and application developers to focus on developing AR applications for languages other than English, incorporate higher-order learning outcomes in the language learning activities, and pursue qualitative investigations.

Background Augmented Reality (AR) is a rapidly emerging technology finding growing acceptance and application in different fields of surgery. Various studies have been performed evaluating the precision and accuracy of AR guided navigation. This study investigates the feasibility of a commercially available AR head mounted device during orthopedic surgery. Methods Thirteen orthopedic surgeons from a Swiss university clinic performed 25 orthopedic surgical procedures wearing a holographic AR headset (HoloLens, Microsoft, Redmond, WA, USA) providing complementary three-dimensional, patient specific anatomic information. The surgeon’s experience of using the device during surgery was recorded using a standardized 58-item questionnaire grading different aspects on a 100-point scale with anchor statements. Results Surgeons were generally satisfied with image quality (85 ± 17 points) and accuracy of the virtual objects (84 ± 19 point). Wearing the AR device was rated as fairly comfortable (79 ± 13 points). Functionality of voice commands (68 ± 20 points) and gestures (66 ± 20 points) provided less favorable results. The greatest potential in the use of the AR device was found for surgical correction of deformities (87 ± 15 points). Overall, surgeons were satisfied with the application of this novel technology (78 ± 20 points) and future access to it was demanded (75 ± 22 points). Conclusion AR is a rapidly evolving technology with large potential in different surgical settings, offering the opportunity to provide a compact, low cost alternative requiring a minimum of infrastructure compared to conventional navigation systems. While surgeons where generally satisfied with image quality of the here tested head mounted AR device, some technical and ergonomic shortcomings were pointed out. This study serves as a proof of concept for the use of an AR head mounted device in a real-world sterile setting in orthopedic surgery.

The application of artificial intelligence technology in higher education has study value and study space. The study mainly used core data collection of the Web of Science database based on SSCI and SCI journals and based on “artificial intelligence” and “higher education” these two subjects retrieval for 2009 to 2019. In addition, It used keyword co-occurrence function and cluster analysis function of CiteSpace Knowledge Graph software for visualization analysis and detailed analysis of study hotspots. The study elaborated on the application of artificial intelligence technology in higher education, and gained study hotspots such as machine learning, augmented reality, and learning management systems, and briefly discussed the impact of artificial intelligence on higher education.

Visual simultaneous localization and mapping (VSLAM) is an essential technique used in areas such as robotics and augmented reality for pose estimation and 3D mapping. Research on VSLAM using both monocular and stereo cameras has grown significantly over the last two decades. There is, therefore, a need for emphasis on a comprehensive review of the evolving architecture of such algorithms in the literature. Although VSLAM algorithm pipelines share similar mathematical backbones, their implementations are individualized and the ad hoc nature of the interfacing between different modules of VSLAM pipelines complicates code reuseability and maintenance. This paper presents a software model for core components of VSLAM implementations and interfaces that govern data flow between them while also attempting to preserve the elements that offer performance improvements over the evolution of VSLAM architectures. The framework presented in this paper employs principles from model-driven engineering (MDE), which are used extensively in the development of large and complicated software systems. The presented VSLAM framework will assist researchers in improving the performance of individual modules of VSLAM while not having to spend time on system integration of those modules into VSLAM pipelines.

Fifth-generation (5G) wireless networks are projected to bring a major transformation to the current fourth-generation network to support the billions of devices that will be connected to the Internet. 5G networks will enable new and powerful capabilities to support high-speed data rates, better connectivity and system capacity that are critical in designing applications in virtual reality, augmented reality and mobile online gaming. The infrastructure of a network that can support stringent application requirements needs to be highly dynamic and flexible. Network slicing can provide these dynamic and flexible characteristics to a network architecture. Implementing network slicing in 5G requires domain modification of the preexisting network architecture. A network slicing architecture is proposed for an existing 5G network with the aim of enhancing network dynamics and flexibility to support modern network applications. To enable network slicing in a 5G network, we established the virtualisation of the underlying physical 5G infrastructure by utilising technological advancements, such as software-defined networking and network function virtualisation. These virtual networks can fulfil the requirement of multiple use cases as required by creating slices of these virtual networks. Thus, abstracting from the physical resources to create virtual networks and then applying network slicing on these virtual networks enable the 5G network to address the increased demands for high-speed communication.

Purpose Emerging holographic headsets can be used to register patient-specific virtual models obtained from medical scans with the patient’s body. Maximising accuracy of the virtual models’ inclination angle and position (ideally, ≤ 2° and ≤ 2 mm, respectively, as in currently approved navigation systems) is vital for this application to be useful. This study investigated the accuracy with which a holographic headset registers virtual models with real-world features based on the position and size of image markers. Methods HoloLens® and the image-pattern-recognition tool Vuforia Engine™ were used to overlay a 5-cm-radius virtual hexagon on a monitor’s surface in a predefined position. The headset’s camera detection of an image marker (displayed on the monitor) triggered the rendering of the virtual hexagon on the headset’s lenses. 4 × 4, 8 × 8 and 12 × 12 cm image markers displayed at nine different positions were used. In total, the position and dimensions of 114 virtual hexagons were measured on photographs captured by the headset’s camera. Results Some image marker positions and the smallest image marker (4 × 4 cm) led to larger errors in the perceived dimensions of the virtual models than other image marker positions and larger markers (8 × 8 and 12 × 12 cm). ≤ 2° and ≤ 2 mm errors were found in 70.7% and 76% of cases, respectively. Conclusion Errors obtained in a non-negligible percentage of cases are not acceptable for certain surgical tasks (e.g. the identification of correct trajectories of surgical instruments). Achieving sufficient accuracy with image marker sizes that meet surgical needs and regardless of image marker position remains a challenge.

© 2021 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)., Introduction: Innovations in 3D spatial technology and augmented reality imaging driven by digital high-tech industrial science have accelerated experimental advances in breast cancer imaging and the development of medical procedures aimed to reduce invasiveness. Presentation of case: A 57-year-old post-menopausal woman presented with screen-detected left-sided breast cancer. After undergoing all staging and pre-operative studies the patient was proposed for conservative breast surgery with tumor localization. During surgery, an experimental digital and non-invasive intra-operative localization method with augmented reality was compared with the standard pre-operative localization with carbon tattooing (institutional protocol). The breast surgeon wearing an augmented reality headset (Hololens) was able to visualize the tumor location projection inside the patient's left breast in the usual supine position. Discussion: This work describes, to our knowledge, the first experimental test with a digital non-invasive method for intra-operative breast cancer localization using augmented reality to guide breast conservative surgery. In this case, a successful overlap of the previous standard pre-operative marks with carbon tattooing and tumor visualization inside the patient's breast with augmented reality was obtained. Conclusion: Breast cancer conservative guided surgery with augmented reality can pave the way for a digital non-invasive method for intra-operative tumor localization.

MySpira is the world first augmented reality (AR) asthma inhaler training app; it enables children to learn asthma keywords, types of inhalers, the preparation of the inhaler and spacer, asthma triggers, and develop skills on how to use an asthma inhaler effectively. All of these are compiled into an enjoyable 20-minute experience, introducing likable characters and tactile interactions, which incentivise the child to progress all the way through the learning experience. This study aimed to compare the efficacy of the MySpira application to traditional educational material (e.g. leaflets and videos) to improve asthma treatment compliance, technique and user engagement amongst 110 schoolchildren, aged 6 to 13. The findings indicate that MySpira surpasses the traditional educational materials, specifically in terms of enjoyment rating, the use of different inhalers’ techniques, particularly in the younger group (aged 6-9 years). Therefore, MySpira may enhance the level of available asthma care in an attempt to resolve one of the major problems facing asthma suffers and their carers, notably effective use of asthma inhalers.

SUMMARY STATEMENT: The optimal strategy for training cardiopulmonary resuscitation (CPR) for healthcare professionals remains to be determined. This scoping review aimed to describe the emerging evidence for CPR training for healthcare professionals.We screened 7605 abstracts and included 110 studies in this scoping review on CPR training for healthcare professionals. We assessed the included articles for evidence for the following topics: training duration, retraining intervals, e-learning, virtual reality/augmented reality/gamified learning, instructor-learner ratio, equipment and manikins, other aspects of contextual learning content, feedback devices, and feedback/debriefing. We found emerging evidence supporting the use of low-dose, high-frequency training with e-learning to achieve knowledge, feedback devices to perform high-quality chest compressions, and in situ team simulations with debriefings to improve the performance of provider teams.

The timely delivery of resource-intensive and latency-sensitive services (e.g., industrial automation, augmented reality) over distributed computing networks (e.g., mobile edge computing) is drawing increasing attention. Motivated by the insufficiency of average delay performance guarantees provided by existing studies, we focus on the critical goal of delivering next generation real-time services ahead of corresponding deadlines on a per-packet basis, while minimizing overall cloud network resource cost. We introduce a novel queuing system that is able to track data packets' lifetime and formalize the optimal cloud network control problem with strict deadline constraints. After illustrating the main challenges in delivering packets to their destinations before getting dropped due to lifetime expiry, we construct an equivalent formulation, where relaxed flow conservation allows leveraging Lyapunov optimization to derive a provably near-optimal fully distributed algorithm for the original problem. Numerical results validate the theoretical analysis and show the superior performance of the proposed control policy compared with state-of-the-art cloud network control., arXiv admin note: text overlap with arXiv:2205.02427

As the Industry 4.0 takes shape, human operators experience an increased complexity of their daily tasks: they are required to be highly flexible and to demonstrate adaptive capabilities in a very dynamic working environment. It calls for tools and approaches that could be easily embedded into everyday practices and able to combine complex methodologies with high usability requirements. In this perspective, the proposed research work is focused on the design and development of a practical solution, called Sophos-MS, able to integrate augmented reality contents and intelligent tutoring systems with cutting-edge fruition technologies for operators' support in complex man-machine interactions. After establishing a reference methodological framework for the smart operator concept within the Industry 4.0 paradigm, the proposed solution is presented, along with its functional and non-function requirements. Such requirements are fulfilled through a structured design strategy whose main outcomes include a multi-layered modular solution, Sophos-MS, that relies on Augmented Reality contents and on an intelligent personal digital assistant with vocal interaction capabilities. The proposed approach has been deployed and its training potentials have been investigated with field experiments. The experimental campaign results have been firstly checked to ensure their statistical relevance and then analytically assessed in order to show that the proposed solution has a real impact on operators' learning curves and can make the difference between who uses it and who does not., 16 pages, 25 figures, 9 tables

International audience; Objective: A craniotomy is the removal of a part of the skull to allow surgeons to have access to the brain and treat tumors. When accessing the brain, a tissue deformation occurs and can negatively influence the surgical procedure outcome. In this work, we present a novel Augmented Reality neurosurgical system to superimpose pre-operative 3D meshes derived from MRI onto a view of the brain surface acquired during surgery. Methods: Our method uses cortical vessels as main features to drive a rigid then non-rigid 3D/2D registration. We first use a feature extractor network to produce probability maps that are fed to a pose estimator network to infer the 6-DoF rigid pose. Then, to account for brain deformation, we add a nonrigid refinement step formulated as a Shape-from-Template problem using physics-based constraints that helps propagate the deformation to sub-cortical level and update tumor location. Results: We tested our method retrospectively on 6 clinical datasets and obtained low pose error, and showed using synthetic dataset that considerable brain shift compensation and low TRE can be achieved at cortical and sub-cortical levels. Conclusion: The results show that our solution achieved accuracy below the actual clinical errors demonstrating the feasibility of practical use of our system. Significance: This work shows that we can provide coherent Augmented Reality visualization of 3D cortical vessels observed through the craniotomy using a single camera view and that cortical vessels provide strong features for performing both rigid and non-rigid registration.

The complexity of wireless and mobile networks is growing at an unprecedented pace. This trend is proving current network control and management techniques based on analytical models and simulations to be impractical, especially if combined with the data deluge expected from future applications such as augmented reality. This is particularly true for software-defined wireless local area networks (SO-WLANs). It is our belief that to battle this growing complexity, future SO-WLANs must follow an artificial intelligence (AI) -native approach. In this article, we introduce aiOS, which is an AI-based platform that builds toward the autonomous management of SD-WLANs. Our proposal is aligned with the most recent trends in in-network AI promoted by the ITU Telecommunication Standardization Sector (ITU-T) and with the architecture for disaggregated radio access networks promoted by the Open Radio Access Network Alliance. We validate aiOS in a practical use case, namely frame size optimization in SD-WLANs, and we consider the long-term evolution, challenges, and scenarios for AI-assisted network automation in the wireless and mobile networking domain.

The increasing number of road accidents and deaths among children in Malaysia is a concern. One of the main causes stems from lack of knowledge and skills in dealing with the complexity and condition of the road system. Recognizing the significance of road safety among students, road safety education has been introduced to primary schoolsby the Ministry of Education in 2008. Initially it was blended into the Malay language subject with the goal to equip the children withthe knowledge and skills about road safety.However, road safety teaching and learning still uses the conventional methods without the use of specialized tools. This paper introduces ROSE courseware, a specialized learning tool developed to meet the needs of the Malaysian primary school curriculum for road safety education. Its learning environment is a blend of virtual reality, augmented reality and interactive multimedia, which enables children to understand and acquire skills related to road safety through interactive, real-time and immersive learning tool. In order to receive the children’s feedbacks pertaining to their experience in interacting with the ROSE courseware, a usability evaluation was conducted. The results show that the children strongly agreed on the usefulness, information quality and interface quality of the ROSE courseware.

This review paper presents a systematic literature review on the use of Augmented Reality (AR) in engineering education, and specifically in student’s spatial ability training, for the last decade. Researchers have explored the benefits of AR, and its application has been of increasing interest in all levels of education. Engineering students tend to have difficulties in acquiring visualization skills, and hence, AR is gaining momentum in enhancing students’ learning achievements. This paper aims to present valuable information to researchers, tutors and software developers of learning technology systems concerning the advantages and limitations of AR in spatial ability training, the incorporation of adaptivity and personalization in AR applications as well as the aspects of spatial ability having been evaluated using AR and the prevalent evaluation methods for AR applications. To this direction, a total of thirty-two (32) studies were reviewed, having been published since 2010. The findings reveal an increase in the number of studies during the last three years. One major conclusion is the improvement of learners’ spatial ability using AR in educational settings, and the noted challenge is the need for more learning content. One research gap that has been identified is the lack of personalization in the developed applications, offering space for future research. Concluding, this area is under-researched, and thus, there is scope for a lot of improvement.

We propose a preprocessing system that transforms the real world into 3D mesh virtual world to provide augmented reality (AR) services. The proposed system uses a monocular RGB camera to obtain sequential images to generate real space. First, we create 3D real-world space composed of point clouds using sequential color images. And then, we segment the objects in each color image and assign an identification number for each object. Also, we execute ‘3D point labeling’, which assigns identification numbers from 2D segmented object to 3D point cloud through a simple projection manner with several parameters. This process could collect 3D point cloud with same the label by as an object. Finally, only the 3D point clouds that have the same identification number by as an object are used to generate 3D mesh. This paper confirmed that 3D mesh could be created using the only general monocular camera without the help of special 3D sensors.

With the rapid technological advancements in recent decades, virtual reality (VR) and augmented reality (AR) technologies have been increasingly adopted to address various challenges in emergency management in the built environments. This paper presents a review of state-of-the-art applications in this rapidly evolving area. A total of 84 relevant articles are identified based on searching in the Web of Science Core Collection and snowballing. These papers are then organized based on a taxonomy developed in this study. Next, a range of VR/AR applications presented in these papers that are aimed to enhance various processes associated with pre-emergency preparedness, responses during emergency and post-emergency recovery are reviewed in detail. The existing VR/AR applications are also described from a human-computer interaction perspective. Finally, current research trends, knowledge gaps and directions for future research are discussed. The findings presented in this paper are expected to provide a synthetic and critical review of state-of-the-art VR/AR applications for emergency management in the built environment and facilitate further advancements in both research and practice in this area.

Education is a place to introduce culture and micro, small and medium enterprises in areas far from the capital. In Technology Education Study Program, Universitas Negeri Jakarta has the vision to produce educators and education personnel in the field of learning to engineer based on academic principles and ethics. Engineering new media learning through augmented reality becomes a tool for lectures introducing SMEs. For this reason, researchers collaborated with material expert lecturers, design experts, media experts, and 30 students in developing augmented reality as a new learning application. The results of the review obtained from the instructional design experts were 3.20, material expert 3,30, and media experts 3.14. Based on the test results obtained from students in the one-to-one stage of 3.55; and at the small group stage of 3.23. Based on the results of the evaluation, it can be concluded that Augmented Reality can be implemented and used to introduce SMEs to college students. Students can use this product as material strengthening in engineering learning using new media applications

The digital transformation of healthcare is changing the medical profession. Augmented/Virtual Reality (AR/VR) and robotics are being increasingly used in different clinical contexts and require supporting education and training, which must begin within the medical school. There is currently a large discrepancy between the high demand and the number of scientifically proven concepts. The aim of this thesis was the conceptual design and structured evaluation of a newly developed learning/teaching concept for the digital transformation of medicine, with a special focus on the influence of surgical teaching.Thirty-five students participated in three courses of the blended learning curriculum "Medicine in the digital age". The 4th module of this course deals with virtual reality, augmented reality and robotics in surgery. It is divided into the following course parts: (1) immersive surgery simulation of a laparoscopic cholecystectomy, (2) liver surgery planning using AR/VR, (3) basic skills on the VR simulator for robotic surgery, (4) collaborative surgery planning in virtual space and (5) expert discussion. After completing the overall curriculum, a qualitative and quantitative evaluation of the course concept was carried out by means of semi-structured interviews and standardised pre-/post-evaluation questionnaires.In the qualitative evaluation procedure of the interviews, 79 text statements were assigned to four main categories. The largest share (35%) was taken up by statements on the "expert discussion", which the students consider to be an elementary part of the course concept. In addition, the students perceived the course as a horizon-widening "learning experience" (29% of the statements) with high "practical relevance" (27%). The quantitative student evaluation shows a positive development in the three sub-competences knowledge, skills and attitude.Surgical teaching can be profitably used to develop digital skills. The speed of the change process of digital transformation in the surgical specialty must be considered. Curricular adaptation should be anchored in the course concept.Die digitale Transformation der Medizin verändert den Beruf des Arztes. Augmented und Virtual Reality (AR/VR) und die Robotik erfahren einen zunehmenden Einsatz in unterschiedlichen klinischen Kontexten und bedürfen einer begleitenden Aus- und Weiterbildung, die bereits im Medizinstudium beginnen muss. Hier besteht aktuell eine große Diskrepanz zwischen dem hohen Bedarf und der Anzahl an wissenschaftlich überprüften Konzepten. Ziel der vorliegenden Arbeit war die Konzeptionierung und strukturierte Evaluation eines neu entwickelten Lern-/Lehrkonzepts zur digitalen Transformation mit Fokus auf die chirurgische Lehre.35 Studierende haben in 3 Kursen des Blended-Learning-Curriculums „Medizin im digitalen Zeitalter“ teilgenommen. Das 4. Modul dieses Kurses thematisiert Virtual Reality, Augmented Reality und Robotik in der Chirurgie. Es gliedert sich in die folgenden Kursteile: (1) immersive Simulation einer laparoskopischen Cholezystektomie, (2) leberchirurgische Operationsplanung mittels AR/VR, (3) Basisfertigkeiten am VR-Simulator für robotische Chirurgie, (4) kollaborative OP Planung im virtuellen Raum und (5) Expertengespräch. Nach Abschluss des Gesamtcurriculums erfolgte eine qualitative und quantitative Evaluation des Kurskonzepts mittels semistrukturierter Interviews sowie anhand von standardisierten Prä-post-Evaluationsfragebögen.Im qualitativen Auswertungsverfahren der Interviews wurden 79 Textaussagen 4 Schwerpunktkategorien zugewiesen. Den größten Anteil (35%) nahmen hierbei Äußerungen zum „Expertengespräch“ ein, das von den Studierenden als elementarer Teil des Kurskonzepts gewertet wurde. Darüber hinaus empfanden die Studierenden den Kurs als horizonterweiterndes „Lernerlebnis“ (29% der Aussagen) mit einem hohen „Praxisbezug“ (27%). Die quantitative Studierendenevaluation zeigt eine positive Entwicklung für die Teilkompetenzen Wissen und Fertigkeiten sowie eine Tendenz zu einer positiven Haltung nach Kursabschluss.Die chirurgische Lehre ist zur Entwicklung digitaler Kompetenzen prädestiniert. Dabei muss die Geschwindigkeit des Veränderungsprozesses der digitalen Transformation im chirurgischen Fachgebiet beachtet und im curricularen Konzept verankert werden.

Purpose of Review This review provides an overview of the most recent robotic ultrasound systems that have contemporary emerged over the past five years, highlighting their status and future directions. The systems are categorized based on their level of robot autonomy (LORA). Recent Findings Teleoperating systems show the highest level of technical maturity. Collaborative assisting and autonomous systems are still in the research phase, with a focus on ultrasound image processing and force adaptation strategies. However, missing key factors are clinical studies and appropriate safety strategies. Future research will likely focus on artificial intelligence and virtual/augmented reality to improve image understanding and ergonomics. Summary A review on robotic ultrasound systems is presented in which first technical specifications are outlined. Hereafter, the literature of the past five years is subdivided into teleoperation, collaborative assistance, or autonomous systems based on LORA. Finally, future trends for robotic ultrasound systems are reviewed with a focus on artificial intelligence and virtual/augmented reality.

In this paper, we designed a registration framework that can be used to develop augmented reality environments, where all the real (including the users) and virtual elements are co-localized and registered in a common reference frame. The software is provided together with this paper, to contribute to the research community. The developed framework allows us to perform a quantitative assessment of interaction and egocentric perception in Augmented Reality (AR) environments. We assess perception and interaction in the peripersonal space through a 3D blind reaching task in a simple scenario and an interaction task in a kitchen scenario using both video (VST) and optical see-through (OST) head-worn technologies. Moreover, we carry out the same 3D blind reaching task in real condition (without head-mounted display and reaching real targets). This provides a baseline performance with which to compare the two augmented reality technologies. The blind reaching task results show an underestimation of distances with OST devices and smaller estimation errors in frontal spatial positions when the depth does not change. This happens with both OST and VST devices, compared with the real-world baseline. Such errors are compensated in the interaction kitchen scenario task. Thanks to the egocentric viewing geometry and the specific required task, which constrain the position perception on a table, both VST and OST have comparable and effective performance. Thus, our results show that such technologies have issues, though they can be effectively used in specific real tasks. This does not allow us to choose between VST and OST devices. Still, it provides a baseline and a registration framework for further studies and emphasizes the specificity of perception in interactive AR.

Human matting, high quality extraction of humans from natural images, is crucial for a wide variety of applications such as virtual reality, augmented reality, entertainment and so on. Since the matting problem is an ill-posed problem, most previous methods rely on extra user inputs such as trimap or scribbles as guidance to estimate alpha value for the pixels that are in the unknown region of the trimap. This phenomenon makes it difficult to be applied to large scale data. In order to solve these problems, we studied the unique role of semantics and details in image matting, and decomposed the matting task into two sub-tasks: trimap segmentation based on high-level semantic information and alpha regression based on low-level detailed information. Specifically, we proposed a novel Cascaded Segmented Matting Network (CSMNet), which uses a shared encoder and two separate decoders to learn these two tasks in a collaborative way to achieve the end-to-end human image matting. In addition, we established a large-scale dataset with 14,000 fine-labeled human matting images. A background dataset is also built to simulate real pictures. Comprehensive empirical studies on above datasets demonstrate that CSMNet could produce a stable and accurate alpha matte without the input of trimap and achieve an evaluation value that is comparable to the algorithm that requires trimap.