Search

Augmented reality applications use object tracking to estimate the pose of a camera and to superimpose virtual content onto the observed object. Today, a number of tracking systems are available, ready to be used in industrial applications. However, such systems are hard to handle for a service maintenance engineer, due to obscure configuration procedures. In this article, we investigate options towards replacing the manual configuration process with a machine learning approach based on automatically synthesized data. We present an automated process of creating object tracker facilities exclusively from synthetic data. The data is highly enhanced to train a convolutional neural network, while still being able to receive reliable and robust results during real world applications only from simple RGB cameras. Comparison against related work using the LINEMOD dataset showed that we are able to outperform similar approaches. For our intended industrial applications with high accuracy demands, its performance is still lower than common object tracking methods with manual configuration. Yet, it can greatly support those as an add-on during initialization, due to its higher reliability.

Current robotic endoluminal interventions require surgeons to hold a proximal joystick to control the distal flexible robot under 2-D X-ray guidance. However, the 2-D X-ray image is not intuitive, which not only increases the risk of surgical misoperation but also the workload of surgeons. Moreover, contact teleoperation exposes surgeons to the potentially infectious environment. To address it, this article proposes an augmented-reality-assisted touchless teleoperated robot for endoluminal intervention, called ARei. It aims to provide immersive experiences with augmented information. The robot integrates perceptual information obtained from an electromagnetic (EM) sensor, a shape sensor, and the virtual anatomy into a head-mounted display (HMD). The touchless teleoperation is used to control the robot, with the assistance of gesture recognition technology. Results show that the mean error of the calibration between the HMD and the EM tracking system is 4.67 mm, and the mean distance error between the points measured by the EM sensor and the points obtained by shape reconstruction with calibration is 5.19 mm (3.05%).

Multi-user mobile Augmented Reality (AR) has been successfully used in various fields as a novel visual interaction technology. But current mainstream wearable device-based and app-based solutions are still facing cross-platform, real-time communication, and intensive computing requirements. Mobile Web technology is envisioned to be a promising supporting technology for cross-platform application of mobile AR especially in 5G networks, which provide pervasive communication and computing resources thereby forming a formidable framework for the practical application of multi-user mobile Web AR. However, the problem of how to use these new techniques properly to achieve efficient communication and computing collaboration is obviously paramount in order for multi-user mobile Web AR to be realized in 5G networks. In this article, we propose the first edge-assisted multi-user collaborative framework for mobile Web AR. Firstly, we propose a heuristic mechanism for efficient communication planning, which allows multi-user interaction synchronization to be achieved. Secondly, we introduce a motion-aware key frame selection mechanism to optimize the computational efficiency of the edge system, and simultaneously alleviate the initialization problem by collaborating with nearby mobile devices using the Device-to-Device communication technique. Experiments are conducted in a real-world 5G network, and the results demonstrate the superiority of our proposed collaborative framework.

Head-mounted loupes can increase the user's visual acuity to observe the details of an object. On the other hand, optical see-through head-mounted displays (OST-HMD) are able to provide virtual augmentations registered with real objects. In this article, we propose AR-Loupe, combining the advantages of loupes and OST-HMDs, to offer augmented reality in the user's magnified field-of-vision. Specifically, AR-Loupe integrates a commercial OST-HMD, Magic Leap One, and binocular Galilean magnifying loupes, with customized 3D-printed attachments. We model the combination of user's eye, screen of OST-HMD, and the optical loupe as a pinhole camera. The calibration of AR-Loupe involves interactive view segmentation and an adapted version of stereo single point active alignment method (Stereo-SPAAM). We conducted a two-phase multi-user study to evaluate AR-Loupe. The users were able to achieve sub-millimeter accuracy ( 0.82 mm) on average, which is significantly ( ) smaller compared to normal AR guidance ( 1.49 mm). The mean calibration time was 268.46 s. With the increased size of real objects through optical magnification and the registered augmentation, AR-Loupe can aid users in high-precision tasks with better visual acuity and higher accuracy.

Simulating shadow interactions between real and virtual objects is important for augmented reality (AR), in which accurately and efficiently detecting real shadows from live videos is a crucial step. Most of the existing methods are capable of processing only scenes captured under a fixed viewpoint. In contrast, this paper proposes a new framework for shadow detection in live outdoor videos captured under moving viewpoints. The framework splits each frame into a tracked region, which is the region tracked from the previous video frame through optical flow analysis, and an emerging region, which is newly introduced into the scene due to the moving viewpoint. The framework subsequently extracts features based on the intensity profiles surrounding the boundaries of candidate shadow regions. These features are then utilized to both correct erroneous shadow boundaries for the tracked region and to detect shadow boundaries for the emerging region by a Bayesian learning module. To remove spurious shadows, spatial layout constraints are further considered for emerging regions. The experimental results demonstrate that the proposed framework outperforms the state-of-the-art shadow tracking and detection algorithms on a variety of challenging cases in real time, including shadows on backgrounds with complex textures, nonplanar shadows, fast-moving shadows with changing typologies, and shadows cast by nonrigid objects. The quantitative experiments show that our method outperforms the best existing method, achieving a 33.3% increase in the average F_{measure}. Coupled with an image-based shadow-casting method, the proposed framework generates realistic shadow interaction results. This capability will be particularly beneficial for supporting AR.

During crisis situations, emergency operators require fast information access to achieve situation awareness and make the best possible decisions. Augmented reality could be used to visualize the wealth of user-generated content available on social media and enable context-adaptive functions for emergency operators. Although emergency operators agree that social media analytics will be important for their future work, it poses a challenge to filter and visualize large amounts of social media data. We conducted a goal-directed task analysis to identify the situation awareness requirements of emergency operators. By collecting tweets during two storms in Germany we evaluated the usefulness of Twitter data for achieving situation awareness and conducted interviews with emergency operators to derive filter strategies for social media data. We synthesized the results by discussing how the unique interface of augmented reality can be used to integrate social media data into emergency control rooms for situation awareness support.

Malfunctions are a major challenge in partially automated driving. During such malfunctions, the driver must be able to adequately take over vehicle guidance without being requested to intervene. This may be particularly difficult in urban areas due to their high complexity and information density. Augmented Reality Head-Up Displays (ARHUDs) may have the potential to support the driver during the monitoring task by providing driving-related information at its required location in the primary field of view. The effects of an ARHUD compared to a Baseline concept in case of malfunctions were investigated in a driving simulation experiment with 52 participants. In a partially automated urban drive, participants experienced a longitudinal and a lateral malfunction in permuted order. The concepts -ARHUD or Baseline- were presented as a between-subject factor. The results showed significantly shorter take-over times when using the ARHUD, resulting in fewer crashes. For those who were able to avoid the crash, no differences in the take-over quality between both concepts were found. There was one difference in visual attention: the attention ratio on the instrument cluster was lower for the ARHUD. In addition, the ARHUD revealed a significantly higher trust and usability rating. However, there were no differences in acceptance and subjective workload between the two concepts. The results showed that the ARHUD has more potential to prevent crashes in the event of malfunctions compared to the Baseline. Nevertheless, the high number of crashes, regardless of the concept, showed the importance of a fallback level for partially automated urban driving.

Egocentric augmented reality (AR) interfaces are quickly becoming a key asset for assisting high precision activities in the peripersonal space in several application fields. In these applications, accurate and robust registration of computer-generated information to the real scene is hard to achieve with traditional Optical See-Through (OST) displays given that it relies on the accurate calibration of the combined eye-display projection model. The calibration is required to efficiently estimate the projection parameters of the pinhole model that encapsulate the optical features of the display and whose values vary according to the position of the user's eye. In this article, we describe an approach that prevents any parallax-related AR misregistration at a pre-defined working distance in OST displays with infinity focus; our strategy relies on the use of a magnifier placed in front of the OST display, and features a proper parameterization of the virtual rendering camera achieved through a dedicated calibration procedure that accounts for the contribution of the magnifier. We model the registration error due to the viewpoint parallax outside the ideal working distance. Finally, we validate our strategy on a OST display, and we show that sub-millimetric registration accuracy can be achieved for working distances of ±100 mm around the focal length of the magnifier.

Refereed/Peer-reviewed This paper presents an Augmented Reality (AR)-based network cabling tutoring system that trains users how to interconnect cables within a Virtual Local Area Network (VLAN) on a physical switching rack. AR arrows combined with text-based instruction and a checklist provided assistance during practical learning. When learners made a mistake, an Intelligent Tutoring System (ITS) identified the source of the mistake and provided real-time feedback using text, AR and text-to-speech mechanisms. The design was motivated by the human-cognitive architecture and its five evolutionary principles (proposed by Sweller and Sweller (2006)). Users performed four consecutive network cabling training tasks with assistance from our ITS. We found that users made fewer errors when the AR cues, text-based instruction and checklist solutions were replaced with summarised information and then removed completely in the final task compared to those who used an identical system with fixed instruction. Cognitive Load Theory (CLT) explains our results by suggesting that the instructional mechanisms become redundant as knowledge increases. Implications of the study are discussed as well as how AR can help facilitate knowledge transfer in the network cabling domain.

The furniture business has developed quite well every year, but the promotions used by the company still use the usual methods, such as using brochures that only display 2D information and images. With so much competition, consumers want something unique and different so they can find out more detailed and real product information offered. Augmented reality is the right technology to meet these consumer needs. Augmented reality is a technology that combines the virtual world with the real world directly. Therefore, an application was designed that functions as a company profile as well as displays 3D rattan chair products using Augmented reality technology on android devices. The method used in this research is the Multimedia Development Life Cycle (MDLC) which includes Concept, Design, Material Collecting, Assembly, Testing, and Distribution. This Augmented reality application requires streaming video taken from a smartphone camera as an input source, then this application will track and detect markers using a tracking system, after the marker is detected, the 3D chair model in the catalog will appear above the marker as if it were a chair model. it's real. the results of the hypothesis test with a significant level of 0.139, which is greater than 0.05, then what is accepted is H1. It can be stated that an augmented reality-based electronic catalog application can increase buying interest in potential consumers of Raffi cushion rotan

Augmented reality is a device in combining objects between 2D and 3D virtual worlds, there's information that has not been implemented directly to the user, then by responding into scope that is 3D on the object with the resulting time at the same time to read in real-time, related to learning media where the teaching and learning process generally uses a book, then develops using projectors and laptops as information technology, besides the development of mobile (cellular) or android-based information technology that can make it easier for learning, especially the introduction of the solar system, the beginning of the solar system introduced various space objects both planets and surrounding space objects, in class VI who received the material as a basis before going to the next level, but the teaching and learning process there are several factors that influence i if it is examined that there are elderly teachers who have difficulty keeping up with technological developments, from learning media support tools such as projectors and teaching aids are still very limited, and not all teachers use information technology as a learning medium, the purpose of which is to make an introduction to AR learning media applications android-based solar system for class VI, based on these 3 problems as a solution an educational application is needed that supports learning activities using android augmented reality for the introduction of the solar system with the marker based tracking method, becomes a marker of 3D objects to bring up from the virtual world to the real which contains information on each of these objects, in other words visualizing the results of the marker directly into the real world, the final result is the creation of an AR application for learning the introduction of the solar system.

The amount of arable land is limited, yet the demand for agricultural food products is increasing. This issue has led to the notion of precision farming, where smart city-based technologies (e.g., Internet of Things, digital twins, artificial intelligence) are employed in combination to cater for increased production with fewer resources. Widely used in manufacturing, augmented reality has demonstrated impactful solutions for information communication, remote monitoring and increased interaction. Yet, the technology has only recently begun to find a footing alongside precision farming solutions, despite the many benefits possible to farmers through augmenting the physical world with digital objects. Therefore, this article reflects on literature discussing current applied solutions within agriculture, where augmented realty has demonstrated a significant impact for monitoring and production. The findings discuss that augmented reality must be coupled with other technologies (e.g., simultaneous localization and mapping algorithms, global positioning systems, and sensors), specifically 9 are identified across 2 application domains (livestock and crop farming) to be beneficial. Attention is also provided on how augmented reality should be employed within agriculture, where related-work examples are drawn from in order to discuss suitable hardware approaches and constraints (e.g., mobility).

The application of augmented reality technology to the partial hepatectomy procedure has high practical significance, but the existing augmented reality navigation system has major drawbacks in the display and registration methods, which result in low precision. The augmented reality surgical navigation system proposed in this study has been improved in the above two aspects, which can significantly improve the surgical accuracy. The use of optical see-through head-mounted displays for imaging displays can prevent doctors from reconstructing the patient's two-dimensional image information in their minds and reduce the psychological burden of doctors. In the registration process, the biomechanical properties of the liver are introduced, and a non-rigid registration method based on biomechanics is proposed and realized by a meshless algorithm. In addition, this study uses the moving grid algorithm to carry out clinical experiments on ex-vivo pig liver for experimental verification. The mark-based interactive registration error is 4.21 ± 1.6 mm, and the registration error is reduced after taking the biomechanical properties of the liver into account, which is − 0.153 ± 0.398 mm. The cutting error of the liver model is 0.159 ± 0.292 mm. In addition, with the aid of the navigation system proposed in this paper, the experiment of ex-vivo pig liver cutting was completed with an error of − 1.164 ± 0.576 mm. As a proof-of-concept study, the augmented reality navigation system proposed in this study improves the traditional image-guided surgery in terms of display and registration methods, and the feasibility of the system is verified by ex-vivo pig liver experiments. Therefore, the navigation system has a certain guiding significance for clinical surgery.

Augmented reality (AR) applications that overlay a user's perception of the real world with digitally generated information are on the cusp of commercial viability. AR has appeared in several commercial platforms like Microsoft HoloLens and smartphones. They extend the user experience beyond two dimensions and supplement a user's normal 3D world. A typical location-based multi-player AR application works through a three-step process, wherein the system collects sensory data from the real world, identifies objects based on their context, and finally, renders information on top of the user's senses. However, because these AR applications frequently exchange data with users, they have exposed new individual and public safety issues. In this paper, we develop ARSpy, a user location tracking system solely based on network traffic information of the user, and we test it on location-based multi-player AR applications. We demonstrate the effectiveness of the proposed scheme via real-world experiments on 12 volunteers and show that we could obtain the geolocation of any target with high accuracy. We also propose three defense methods to mitigate these side channel attacks. Our results reveal a potential security threat in location-based multi-player AR applications and serve as a critical security reminder to AR users.

Augmented reality (AR) has huge potential as a science outreach tool, especially around palaeontology where it is possible to bring extinct animals to life. This paper shares our experiences as academic geoscientists in developing a series of AR applications during a three-year period. We do not focus on the technical issues of app design and coding but on the lessons learnt during this time which may be of value to others, whether academics, event coordinators or museum curators interested in using digital technology to enhance user experience or convey educational messages. We identify eight issues that are worth keeping in mind when commissioning, developing, and deploying AR apps for use as a science communication tool. The issues are: (1) download barriers; (2) ageing novelty of AR; (3) the ‘stickiness’ of an app tends to be moment-locations specific; (4) maintenance of bespoke apps; (5) ‘boom and lapse’ cycles in funding; (6) difficulty of embedding educational content; (7) priority conflict between users and event organisers; and (8) the quality of the animation is key.

The development of Smart Home Controllers has seen rapid growth in recent years, especially for smart devices, that can utilize the Internet of Things (IoT). However, a large portion of the household devices and appliances already in use, are not IoT enabled, and therefore, requires their default control mechanisms for the devices to operate. This paper proposes a smart appliance controller that uses Augmented Reality, MQTT, and other up-to-date platforms to control aftermarket home appliances in the most efficient manner. The proposed work integrates mobile AR with IoT, to control household appliances, with the help of infrared (IR) signals. The characteristics of the system are evaluated through a series of tests and performance measures. The results of the test highlight th quick response time of MQTT for the implementation of a Home Automation System, when compared to the request-reply protocol: CoAP (4 times as fast).

Mutual occlusion is an essential feature for augmented reality (AR) displays for allowing the virtual content to be clearly perceived under an excessively illuminated environment. Although a few works have been done to facilitate the performance of occlusion-capable optical see-through augmented reality (OC-OST-AR) displays, the realization of mutual occlusion in a wide field-of-view (FOV) is still challenging. Divergent from typical hard-edge occlusion and soft edge-occlusion designs, we propose the paired-ellipsoidal-mirror (PEM) structure. The proposed system is allowed to support either hard-edge occlusion or enhanced soft-edge occlusion in a wide FOV by optionally fixing a spatial light modulator (SLM) before the entrance pupil or at an inner focal plane. The numerical aperture (NA) of the system is efficiently increased by the combination of paired ellipsoidal mirror imaging and aperture stop restriction. With proof-of-concept prototypes built, virtual display in a FOV of H160°×V74° and mutual occlusion in a FOV of H122°×V74° are demonstrated with a basic design, respectively. Furthermore, a mixed FOV of H95.3°×V52.9° is demonstrated by an optimized design with vertical parallax reduction and virtual display improvement.

As augmented reality technologies develop, real-time interactions between objects present in the real world and virtual space are required. Generally, recognition and location estimation in augmented reality are carried out using tracking techniques, typically markers. However, using markers creates spatial constraints in simultaneous tracking of space and objects. Therefore, we propose a system that enables camera tracking in the real world and visualizes virtual visual information through the recognition and positioning of objects. We scanned the space using an RGB-D camera. A three-dimensional (3D) dense point cloud map is created using point clouds generated through video images. Among the generated point cloud information, objects are detected and retrieved based on the pre-learned data. Finally, using the predicted pose of the detected objects, other information may be augmented. Our system estimates object recognition and 3D pose based on simple camera information, enabling the viewing of virtual visual information based on object location.

Interaction design for Augmented Reality (AR) is gaining attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015–2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of Human-City Interaction and related technological approaches, followed by reviewing the latest trends of information visualization, constrained interfaces, and embodied interaction for AR headsets. We highlight under-explored issues in interface design and input techniques that warrant further research and conjecture that AR with complementary Conversational User Interfaces (CUIs) is a crucial enabler for ubiquitous interaction with immersive systems in smart cities. Our work helps researchers understand the current potential and future needs of AR in Human-City Interaction.

The visual representation of a product and the role of visualization have recently become a central issue in design research. By enriching a real scene with computer generated objects, Augmented Reality, has proven itself as a valuable Human-Computer Interface in numerous application areas, such as medicine, military, entertainment and manufacturing. In this paper we propose an approach by which a user can create own 3D augmented reality scenes that enable interaction between the real world and virtual assembly's components, while including an animation at the same time. For this purpose, BuildAR Pro software is employed using marker-based camera tracking, while assembly design is obtained with standard CAD system SolidWorks. The animations are developed in 3ds max software package in order to save the assembly as .ive file format, which is helpful to increase the performance of scene rendering and/or viewing.

Image-guided therapies have been on the rise in recent years as they can achieve higher accuracy and are less invasive than traditional methods. By combining augmented reality technology with image-guided therapy, more organs, and tissues can be observed by surgeons to improve surgical accuracy. In this review, 233 publications (dated from 2015 to 2020) on the design and application of augmented reality-based systems for image-guided therapy, including both research prototypes and commercial products, were considered for review. Based on their functions and applications. Sixteen studies were selected. The engineering specifications and applications were analyzed and summarized for each study. Finally, future directions and existing challenges in the field were summarized and discussed.

Purpose The purpose of this paper is to introduce a digitalization model (DM) for building construction courses in architectural education as a response to the recent emerging technologies in the era of digital transformations. This DM is developed and applied through augmented reality (AR) technologies to boost perception, understandings and ability to solve building construction details. Design/methodology/approach Based on a thorough review of recent technologies like AR, virtual reality (VR), building information modelling (BIM) and their applications in architectural education, the methodology involves the generation of a model, its application and evaluation. The model is based on the integration of BIM and AR which is applied into a third year “Building Construction Project” course. Each student has designed a residential building and accordingly prepared the construction drawings by adapting the DM. An online survey –based on Technology Acceptance Model – is conducted to evaluate the DM by quantitative data analysis from SPSS and Excel. Findings The key findings of the study include the following items: determination of the proper digital tools, the definition of the steps and workflow based on the building project phases to develop construction drawings and define precise details effectively. By the help of this, the DM is generated and applied. According to the survey and results, the DM which involves BIM-based AR is considered as beneficial, highly motivating and providing better perception on construction details. Originality/value Implication of AR/VR technologies is frequently seen in design studios, whereas building construction courses state its traditional approach. However, there is a huge potential in the digitalization of building construction education by increasing the perception of students together with the increased level of communication. The study aims to close the gap of digitalization by proposing a DM, which brings a systematic approach considering each phase of building construction project as conceptual, schematic, design development and construction documents by using BIM integrated AR. Moreover, the novel model specifically brings a new approach by generation of QR codes for construction details to embed videos or simulations into the two-dimensional drawing sheets. Furthermore, the DM proposes a new approach to satisfy emerging needs and requirements of the Architecture Engineering Construction industry.

In augmented reality, it is important to achieve visual consistency between inserted virtual objects and the real scene. As specular and transparent objects can produce caustics, which affect the appearance of inserted virtual objects, we herein propose a framework for differential rendering beyond the Lambertian-world assumption. Our key idea is to jointly optimize illumination and parameters of specular and transparent objects. To estimate the parameters of transparent objects efficiently, the psychophysical scaling method is introduced while considering visual characteristics of the human eye to obtain the step size for estimating the refractive index. We verify our technique on multiple real scenes, and the experimental results show that the fusion effects are visually consistent.

Purpose Augmented reality (AR) integrates the digital world with the real world and thus, provides a real-time experience to the users. With AR, the immediate surroundings become a learning platform for the users. The perception of the products has been enhanced many times with AR; thus, enriching user experience and responsiveness. The purpose of this paper is to bring forth the basics of AR and provide an overview of the research work carried out by researchers in the implementation of AR in different sectors. Design/methodology/approach This paper summarizes the usefulness of AR in different industries. The authors have identified the peer-reviewed research publications from Web of Science, Scopus, Google Scholar, etc. The selection of literature has been made based upon the significance of AR in recent times. The industries/sectors where AR has been implemented successfully have been considered for this paper. The paper has been divided into various sections and subsections to bring more clarity to the readers. Findings This paper presents a brief and a precise information on Industry 4.0 and AR. The basic working of AR system and its implications have also been discussed. The preference of AR over virtual reality (VR) has also been deliberated in this paper. The authors have presented the usefulness of AR in different sectors such as smart factories, ship yard building, online shopping, surgery and education. This paper discusses the AR-ready procedures being followed in these sectors. Originality/value AR has been an add-on to VR systems. The processes in industries have become very handy and informative with AR. Because the application of AR in different sectors has not been discussed in a single paper; thus, this work presents a systematic literature review on the applications of AR in different sectors/industries.

A software development process comprises all the steps necessary to produce a software product. This research proposes a new process for developing augmented reality serious games (ARSGs), which comprises three phases: analysis, configuration, and generation. The analysis phase involves examining the standard project structure of a typical serious game to identify the applicable actions to which AR content can be added. The configuration phase involves configuring the AR features and the AR library to be integrated into the game structure. Finally, the generation phase entails inserting the AR code in the game structure, as well as all the files necessary to in-corporate AR features in the game. Our process allows generating ARSGs step by step, both easily and rapidly. To implement this process, we propose an ARSG development software architecture. The underlying assumption of this architecture is that software development tools are essential validation elements of the software generation process. By implementing our ARSG development process through a software architecture, we provide users with a formal, automated method for creating ARSGs. The ARSG development process and the architecture, collectively called ZeusAR, are not dependent on specific software development technologies and/or programming languages. To perform a proof-of-concept of our process, we developed an ARSG generator tool based on the proposed architecture and conducted a case study in which geometry ARSGs are developed to help high school students learn about geometric shapes and their properties. We conducted a qualitative evaluation of the ZeusAR tool through a System-Usability-Scale (SUS) based survey, which was administered to a group of geometry teachers to evaluate the tool’s characteristics, such as ease of use and ease of configuration. Additionally, a group of software developers and professors assessed the performance of the ZeusAR tool in terms of game development time.

With the rapid developments and improvements in mobile technologies, mobile devices have become one of the primary interfaces of augmented reality technologies. In this study, we explore how user groups (satisfied vs. dissatisfied users) and domains (game vs. non-game domain) affect user experience, including perceived usefulness, usability, and affection using two-factor theory and analysis of review comments. We employ two linguistic approaches for processing the content of the comments. The results of a series of analyses of variance indicate that both user groups and domains significantly affect user-perceived usability and affection, while there is no effect of user groups on the perceived usefulness of mobile applications. Based on the findings of the study, notable implications and a few limitations are presented.

Proper occlusion based rendering is very important to achieve realism in all indoor and outdoor Augmented Reality (AR) applications. This paper addresses the problem of fast and accurate dynamic occlusion reasoning by real objects in the scene for large scale outdoor AR applications. Conceptually, proper occlusion reasoning requires an estimate of depth for every point in augmented scene which is technically hard to achieve for outdoor scenarios, especially in the presence of moving objects. We propose a method to detect and automatically infer the depth for real objects in the scene without explicit detailed scene modeling and depth sensing (e.g. without using sensors such as 3D-LiDAR). Specifically, we employ instance segmentation of color image data to detect real dynamic objects in the scene and use either a top-down terrain elevation model or deep learning based monocular depth estimation model to infer their metric distance from the camera for proper occlusion reasoning in real time. The realized solution is implemented in a low latency real-time framework for video-see-though AR and is directly extendable to optical-see-through AR. We minimize latency in depth reasoning and occlusion rendering by doing semantic object tracking and prediction in video frames.

Interaction is one of the important topics to be discussed since it includes the interface where the end-user communicates with the augmented reality (AR) system. In handheld AR interface, the traditional interaction techniques are not suitable for some AR applications due to the different attributes of handheld devices that always refer to smartphones and tablets. Currently interaction techniques in handheld AR are known as touch-based technique, mid-air gesture-based technique and device-based technique that can led to a wide discussion in related research areas. However, this paper will focus to discover the device-based interaction technique because it has proven in the previous studies to be more suitable and robust in several aspects. A novel device-based 3D object rotation technique is proposed to solve the current problem in performing 3DOF rotation of 3D object. The goal is to produce a precise and faster 3D object rotation. Therefore, the determination of the rotation amplitudes per second is required before the fully implementation. This paper discusses the implementation in depth and provides a guideline for those who works in related to device-based interaction.

The translator application from Banjar Language to Indonesian is called BandoAR and vice versa from Indonesian to Banjar Language which is called NdoBAR. Both applications utilize Mobile Augmented Reality technology with the Optical Character Recognition (OCR) Method for word recognition in the detected image. This application is recommended and used to help tourists visiting the city of Banjarmasin to be able to understand the language used by local people as a means of communication and for the general public who want to know the peculiarities of the Banjar language itself. The purpose of this study was to evaluate the user experience of the BandoAR and NdoBAR applications. The method used is UX Honeycomb, which has seven aspects to assess the user experience of an application. A total of 50 respondents were presented to assess the two applications. The results showed that the BandoAR application had an average UX Honeycomb value of 4.91 with a Very Strongly Agree predicate and for the NdoBAR application, an average value of UX Honeycomb was 4.89 with a Very Strongly Agree predicate. Both applications have fulfilled aspects of the user experience. However, we need some fixes for the shortcomings of both apps, to continue to improve interactions, better user experience, and other smart capabilities.

Augmented Reality (AR) applications can enable geographically distant users to collaborate using shared video feeds or interactive 3D holograms, and may be particularly useful in the socially distant context of the Covid-19 pandemic. However, a good user experience is key for their success and could be negatively impacted by network impairments, which are an inevitable occurrence in today's best-effort Internet. In this paper, we present the findings of an empirical user study, aimed at understanding the effects of network outages, on user experience and behavior, in a collaborative AR task. We highlight how network outages affected users in different ways depending on their role in the collaborative task, and how giving users explicit information about poor network conditions helped them deal with some of these negative effects. Furthermore, we report the strategies that users themselves adopted, to deal with outages, such as batching instructions, or shifting to a different spatial referencing style when communicating with their partners. Lastly, based on our findings, we present some design implications for future remote-collaborative AR applications.

Multimodal interaction has become a recent research focus since it offers better user experience in augmented reality (AR) systems. However, most existing works only combine two modalities at a time, e.g., gesture and speech. Multimodal interactive system integrating gaze cue has rarely been investigated. In this article, we propose a multimodal interactive system that integrates gaze, gesture, and speech in a flexibly configurable AR system. Our lightweight head-mounted device supports accurate gaze tracking, hand gesture recognition, and speech recognition simultaneously. The system can be easily configured into various modality combinations, which enables us to investigate the effects of different interaction techniques. We evaluate the efficiency of these modalities using two tasks: the lamp brightness adjustment task and the cube manipulation task. We also collect subjective feedback when using such systems. The experimental results demonstrate that the Gaze+Gesture+Speech modality is superior in terms of efficiency, and the Gesture+Speech modality is more preferred by users. Our system opens the pathway toward a multimodal interactive AR system that enables flexible configuration.

Pengetahuan mengenai perlengkapan kemiliteran tidak hanya diperoleh melalui pameran saja, dengan adanya perkembangan teknologi informasi bisa diperoleh melalui aplikasi. Penggunaan aplikasi augmented reality dapat dijadikan contoh dalam pengetahuan perlengkapan militer secara tiga dimensi. Perancangan ini bertujuan memberikan pengetahuan kepada masyarakat umum dengan menggunakan teknologi Augmented Reality. Perancangan ini menggunakan metode ADDIE dan algoritma FAST Corner dan Lucas Kanade. Perancangan ini menghasilkan sebuah aplikasi yang berisi pengenalan perlengkapan militer yang terdiri dari baret, topi boonie, topi patroli, helm, kacamata, tas besar, P3K (Pertolongan Pertama Pada kecelakaan), pisau, senjata jauh dan senjata menengah. Informasi singkat mengenai masing-masing perlengkapan militer yang dirancang secara menarik agar masyarakat umum mudah memahami dalam menggunakan aplikasi yang dapat dijalankan pada platform Android.

Pada saat ini diseluruh bagian dunia sedang terlanda sebuah virus yaitu COVID-19 yang membuat setiap orang tidak dapat berpergian ke luar negri dengan mudah. Namun di era sudah canggih ini semua informasi dapat disajikan secara digitalisasi mulai dalam bentuk tulisan maupun gambar dan dapat juga ditampilkan berbentuk objek 3 dimensi dibantu dengan Augmented Reality untuk menampilkannya. Peneleitian ini memanfaatkan teknologi Augmented Reality digunakan untuk alat pengenalan landmark dari negara Asia Tenggara agar tampilan objek akan menjadi lebih atraktif dan dapat mengetahui informasi tentang landmark dari setiap negara dengan mudah. Dalam penelitian ini aplikasi Augmented Reality dibuat menggunakan FAST Corner Detection (FCD) algoritma dan Natural Feature Tracking (NFT) untuk mendeteksi marker. Pengujian dari aplikasi ini dilakukan pada tiga perangkat smartphone android, pada sudut kemiringan 20° - 90° ketiga smartpone dapat mendeteksi marker menggunakan kameranya dan menampilkan objek 3 dimensi sesuai dengan landmark yang dipilih pada layar smartphone. Untuk pengukuran jarak maximal yang dapat dibaca oleh handphone rata-rata ±75cm dan jarak minimun pendeteksian yaitu ±10cm.

The objective of the present study is to produce a learning media product of health protocol implementation to children entitled "Awas Ada Virus Corona". This media utilized RD method and took several steps, namely preliminary research and data collection, research planning, product development, expert validation, product revision and final product. Feasibility testing is carried out by media experts by examining the multimedia construction domain consist of 3 aspects, namely guidance and information, program performance as well as systematic, aesthetic, and design aspects. Furthermore, a filled questionnaire by media experts was used to collect the data needed. Meanwhile, the descriptive statistic was used to analyzed the data gathered. The result of the present study is an application that can be used by children entitled " Awas Ada Virus Corona " as an educational media in implementing 3M of health protocol, namely wearing masks, washing hands, and maintaining a safe distance. The addition of Augmented reality technology with a live coloring feature is added in this application in reading markers to increase children's attractiveness. This augmented reality technology is applied in order to avoid the children only focus on gadgets, so children feel happy in using the familiar media and still interacting with the environment.

Augmented Reality is a type of interactive technology that combines real objects and virtual objects at the same time and is displayed on the screen. Augmented Reality has been applied as a way of working based on image detection and it is commonly called a marker. By using the smartphone's camera and then detects the marker that has been printed. Augmented Reality is widely used in various purposes, the example is for advertising or promotion. In advertising, Augmented Reality can be used as a promotion or information that can make it more interesting. This research aims to apply AR technology to the campus promotion media on the University of Panca Marga's new student admission brochure. The stages taken in this research are literature study as well as observation, design, and implementation stages. The application of Augmented Reality was built using Unity 3D software, Android SDK, Vuforia SDK, and Sweet Home 3D. The results of the application of Augmented Reality technology can display 3D virtual objects, 2D, and video using the new student admission brochure used as a marker. The marker that is detected by the camera on the Android smartphone can display 3D UPM buildings, location plans, UPM profile videos, and faculty digital brochures. So users can see how the campus buildings are in real-time as well as some other information that has been provided. Based on the tests, all features can run well.

In recent years, the development of Augmented Reality (AR) frameworks made AR application development widely accessible to developers without AR expert background. With this development, new application fields for AR are on the rise. This comes with an increased need for visualization techniques that are suitable for a wide range of application areas. It becomes more important for a wider audience to gain a better understanding of existing AR visualization techniques. In this article we provide a taxonomy of existing works on visualization techniques in AR. The taxonomy aims to give researchers and developers without an in-depth background in Augmented Reality the information to successively apply visualization techniques in Augmented Reality environments. We also describe required components and methods and analyze common patterns.

Public bus transportation has become an integral part of society, but the disrup-tion of bus services is one of the major concerns. This project presents the devel-opment of Smart Bus Transportation using Augmented Reality (TRANSPAR) that was developed on a mobile application. One of the major issues with public transportation is on real-time responsiveness. Most bus schedules are presented online but customers still faced many failures. Some bus schedules are not updat-ed when changes happened through time. Some existing bus schedules system is fixed to the bus stations. This research is to identify the bus schedules and its routes characteristics. A 3D AR animation based on identified characteristics was designed using the Unity 3D image marker detection on a mobile Android plat-form. A smartphone application was developed using Vuforia and Google Fire-base. TRANSPAR shows an AR mobile application for acquiring the bus time-tables. The phone camera is applied for marker image detection and scanning the bus station’s images. AR and normal image scanner were designed. Google Fire-base Database is used to retrieve and store each timetable data for every bus sta-tion. Analysis of interactivity and benefits of TRANSPAR shows about 90% agreed on the use of AR and more than 76% agreed on its functionality based on 50 taken samples. This shows a positive impact on the designed TRANSPAR. The research is significant to encourage and experience the public with new tech-nological application for public transportation and it impact the society.

Traditional neurosurgical craniotomy primarily uses two‐dimensional cranial medical images to estimate the location of a patient’s intracranial lesions. Such work relies on the experience and skills of the doctor and may result in accidental injury to important intracranial physiological tissues. To help doctors more intuitively determine patient lesion information and improve the accuracy of surgical route formulation and craniotomy safety, an augmented reality method for displaying neurosurgery craniotomy lesions based on feature contour matching is proposed. This method uses threshold segmentation and region growing algorithms to reconstruct a 3‐D Computed tomography image of the patient’s head. The augmented reality engine is used to adjust the reconstruction model’s relevant parameters to meet the doctor’s requirements and determine the augmented reality matching method for feature contour matching. By using the mobile terminal to align the real skull model, the virtual lesion model is displayed. Using the designed user interface, doctors can view the patient’s personal information and can zoom in, zoom out, and rotate the virtual model. Therefore, the patient’s lesions information can be visualized accurately, which provides a visual basis for preoperative preparation.

Aiming at the problems of large data volume, long calculation time, and information feedback speed in traditional virtual augmented reality-based scenes, this paper constructed a virtual augmented reality scene based on 5G technology. First, based on the research of camera models and related calibration techniques, this article discusses the coordinate conversion and registration between virtual scenes and real scenes in augmented reality systems based on natural feature points. Secondly, the control mechanism of data synchronization and transmission are realized by using 5G communication technology. Then, through the approximate nearest neighbour search of the descriptor, the feature points are quickly matched. Finally, by tracking the QR-code logo black frame, the global holography matrix virtual real registration method relative to the initial image is used to calculate the camera’s posture and the fast tracking registration of the scene is realized. The experimental results prove that the scene constructed in this paper has the characteristics of real-time, wide field of view, high immersion and flexible expansion, which not only meets the requirements of matching accuracy, but also has the advantages of small calculation amount and fast calculation speed, and has good robustness and real-time.

Aiming at the problem of real-time augmented reality tracking registration of motion blurred template matching image construction model, this paper proposes a template matching target tracking algorithm based on improved efficient second-order minimization (ESM). Based on ESM algorithm, an improved efficient second-order minimization algorithm is used to track the motion template. Firstly, the image acquisition and noise reduction preprocessing are carried out, and the image acquisition model of high-speed moving target and the edge feature detection algorithm of moving feature points are designed. The gray-scale histogram denoising method is used to suppress the edge fuzzy set of dynamic feature points in the image of high-speed moving target. The feature extraction and matching are modeled, and the selection of feature points and the search algorithm are optimized. Then the block detection method is used to reconstruct the region of high-speed moving target, and the edge contour detection and feature point matching of the moving target are carried out in the affine invariant region. The image block template matching method is used to realize the real-time augmented reality tracking registration of high-speed moving target. The simulation results show that the proposed method has good effect and superior performance in real-time augmented reality tracking registration of moving targets.

The present study reviews the augmented reality application in the education area. Justification of the use is to improve the learning motivation and outcome of students through the use of a deep learning approach. A deep learning approach is associated with better academic performances. Augmented reality provides a tool to motivate learning and understanding in the student. In the medical domain, augmented reality may serve as replacement of cadavers especially during online learning. The main aim of the current review is to reflect the use of augmented reality as a teaching and learning tool to motivate and promote the deep learning approach process which includes life-long learning and possible research areas that warrant and assess the impact of augmented reality on academic performance. Augmented reality may be a tool in complementing the learning aids of the conventional methods of teaching and learning.

With the rapid development and extensive promotion of computer and Internet technology, users are more and more dependent on the network. The traditional information management system has been unable to meet the actual needs of a large number of concurrent access and fast storage. Therefore, it is very important to establish a unified cloud application platform for centralized management and unified identity of users in different systems. Through the combination of user experience testing technology, Internet of Things technology and cloud computing technology, the automatic user experience test of network application is realized, and the test report is generated. Managers can view the report through the report display interface provided by the user experience test cloud platform. Before the information system goes online, it will greatly improve the user experience quality of the information system by using the user experience test cloud platform. The designed pipeline integrates the augmented system and the Social Internet of Thing system to provide the efficient framework. Through the testing, the performance of the model is validated, the accuracy has reached the state-of-the-art level.

Purpose The COVID-19 virus outbreak began in December 2019 and rapidly spread to every continent on Earth. The analysts have predicted that COVID-19 and other similar pandemics will continue in the coming decade and badly affect offline businesses. As a result, the offline platform is also shifting to the online platform and online demands are increasing daily. The traditional two-dimensional E-Commerce websites are designed to provide simple, browser-based interfaces to allow users to access available products and services. Whilst virtual representations are an essential consideration in establishing trust, most virtual representation sites fall short in mimicking real-life human representation. This paper aims to focus on three-dimensional (3D) E-Commerce technology that presents how virtual reality (VR) and augmented reality (AR) can help deal with limitations and improve E-Commerce operations. It is built as an internet-only tool, a person-centred shopping assistant created following user-centred design principles to be used on various computing platforms, including desktop and mobile devices. The paper shows how VR and AR can offer more precise product information in 3D E-Commerce environments. The virtual store experience is also enhanced by an AR assistant that helps the users by giving them all the required information in audio form or using its avatar. Design/methodology/approach Implementation of VR and AR in E-Commerce will increase customer satisfaction. Sub hypothesis – to study the implementation of VR in E-Commerce. To study the implementation of AR in E-Commerce. To study the inclusion of E-Commerce sites in an open-world game. To study the customer satisfaction of users using VR stores. Findings The scope of work is concentrated on the urban Indian market especially targeting the country’s youth who are already or ready to indulge in VR such as video games, cinema and other activities (Mattsson and Barkman, 2019). This demography is more open to learning and using VR. The primary segment of E-Commerce that we are concentrating upon is fashion. Here, the regular user needs to have more immersed knowledge about the product rather than just the written information like how would they look in a dress or will the size available on the website fit me or not. Originality/value A perfect system does not exist in the world. A terrible disease has landed on the planet. Very soon, it will be impossible to escape from this current situation. The effects of this plague have been felt in every sector of the world. The researchers also claim that physical stores will continue to exist. There will never be anything that replaces the ability to hold and use products or have personal face-to-face interactions with retail professionals. For the time being, brick-and-mortar retail is having a difficult time, but immersive technology is starting to be used to enhance the in-store experience. The good news is that this should help retailers increase their chances of survival. However, the melody of 3D E-Commerce is it would help out the in-store experience.

Kemajuan teknologi dimasa sekarang sangatlah pesat, Salah satunya dalam bidang media pembelajaran atau edukasi.Teknologi Augmented Reality (AR) Merupakan sebuah penggabungan dari digital konten yang telah di buat di komputer dari dengan dunia nyata.Penelitian ini bertujuan untuk memberi pengenalan tentang tumbuhan sehingga dapat mengetahui informasi tentang tanaman tersebut. Penelitian ini menggunakan sebuah metode yaitu natural feature tracking serta menggunakan sebuah sistem pengembangan yaitu MDLC( Multimedia Development Life Cycle) dan memakai algoritma Fast Corner Detection dan Natural Feature Tracking, Sehingga menghasilkan sebuah aplikasi yang dapat menampilkan objek 3D tentang tumbuhan, dan terdapat penjelasan singkat mengenai tumbuhan, Aplikasi ini dapat menggunakan sistem operasi Android, Pengujian aplikasi menggunakan 3 handphone android yaitu Samsung A2018 dengan versi andorid 9,Oppo A9 2019 dan Samsung A51 2020 bahwa pada saat kemiringan

Augmented reality (AR) is a new and emerging technology that could benefit from evaluating its usability to better the user’s experience with the device or application. This is often done through usability testing and heuristic evaluations. However, AR technology presents some challenges when completing these usability evaluations. Practitioners need to keep in mind the hardware limitations of AR devices that may not be present with other computerized technology, consistency of the users’ environment plays a larger role in the AR experience, recognize that a novelty effect may occur and affect subjective scores, and choose heuristic sets that will best evaluate AR applications. Practitioners need to be aware of these challenges and overcome them to accurately assess the usability of these products to gain insights about what should be changed to make the overall experience with the product better.

Nowadays, breakthrough technologies, such as virtual reality (VR), augmented reality (AR), and Blockchain, have definitively attracted the attention of a huge number of investors worldwide. Although, at first glance, Blockchain (traditionally used for financial services) seems to have little to none to share with VR and AR (originally adopted for entertainment), in the last few years several use cases started to appear showing effective ways to integrate these technologies. In this article, an overview of opportunities investigated by current solutions combining VR, AR, and Blockchain will be discussed, showing how they allowed both companies and academic researchers cope with issues affecting traditional services and products in a rather heterogenous set of application domains. Opportunities that could foster the convergence of these technologies and boost them further are also discussed.