Search

The internet-of-things (IoT) refers to the growing field of interconnected pervasive computing devices and the networking that supports smart, embedded applications. The IoT has multiple human-computer interaction challenges due to its many formats and interlinked components, and central to these is the need to provide sensory information and situational context pertaining to users in a more human-friendly, easily understandable format. This work addresses this by applying mixed reality toward expressing the underlying behaviors and states internal to IoT devices and IoT-enabled objects. It extends the authors’ previous research on IoT Avatars (mixed reality character representations of physical IoT devices), presenting a new head-mounted display framework and interconnection architecture. This contributes i) an exploration of mixed reality for smart spaces, ii) an approach toward expressive avatar behaviors using fuzzy inference, and iii) an early functional prototype of a hybrid physical and mixed reality IoT-enabled object. This approach is a step toward new information presentation, interaction, and engagement capabilities for smart devices and environments.

The furniture business has developed quite well every year, but the promotions used by the company still use the usual methods, such as using brochures that only display 2D information and images. With so much competition, consumers want something unique and different so they can find out more detailed and real product information offered. Augmented reality is the right technology to meet these consumer needs. Augmented reality is a technology that combines the virtual world with the real world directly. Therefore, an application was designed that functions as a company profile as well as displays 3D rattan chair products using Augmented reality technology on android devices. The method used in this research is the Multimedia Development Life Cycle (MDLC) which includes Concept, Design, Material Collecting, Assembly, Testing, and Distribution. This Augmented reality application requires streaming video taken from a smartphone camera as an input source, then this application will track and detect markers using a tracking system, after the marker is detected, the 3D chair model in the catalog will appear above the marker as if it were a chair model. it's real. the results of the hypothesis test with a significant level of 0.139, which is greater than 0.05, then what is accepted is H1. It can be stated that an augmented reality-based electronic catalog application can increase buying interest in potential consumers of Raffi cushion rotan

In the recent years, medical education is facing various difficulties. Students are struggling to visualize the internal organs and bones, it is insufficient cadaver for dissection demonstration, there are lack of equipment for training, and not enough opportunity for hands-on training. Technologies have been utilized to supplement teaching in medical education to overcome the mentioned difficulties. Augmented Reality in particular mobile Augmented Reality has shown a rise in the application of medical education in this decade. This paper aims to do a narrative review to study the overall progress and development of mobile Augmented Reality application in medical education from year 2010 to 2020. From the review done, we found that the application of mobile Augmented Reality in medical education is feasible, it is well accepted by medical students. We also identified current research gaps and potential future development of mobile Augmented Reality in medical education. It was found that there is lack of research done to study long term effect on the users. Interaction between virtual object and real-world object was lacking in some applications. More refinement and update are needed on flow, media and design of Augmented Reality application. There is also essential to study the influence of marker size or surface material.

PURPOSE OF REVIEW: While limited to case reports or small case series, emerging evidence advocates the inclusion of smartphone-interfacing mobile platforms and wearable technologies, consisting of internet-powered mobile and wearable devices that interface with smartphones, in the orthopaedic surgery practice. The purpose of this review is to investigate the relevance and impact of this technology in orthopaedic surgery. RECENT FINDINGS: Smartphone-interfacing mobile platforms and wearable technologies are capable of improving the patients’ quality of life as well as the extent of their therapeutic engagement, while promoting the orthopaedic surgeons’ abilities and level of care. Offered advantages include improvements in diagnosis and examination, preoperative templating and planning, and intraoperative assistance, as well as postoperative monitoring and rehabilitation. Supplemental surgical exposure, through haptic feedback and realism of audio and video, may add another perspective to these innovations by simulating the operative environment and potentially adding a virtual tactile feature to the operator’s visual experience. SUMMARY: Although encouraging in the field of orthopaedic surgery, surgeons should be cautious when using smartphone-interfacing mobile platforms and wearable technologies, given the lack of a current academic governing board certification and clinical practice validation processes.

This article presents a novel extended reality (XR) and deep-learning-based Internet-of-Medical-Things (IoMT) solution for the COVID-19 telemedicine diagnostic, which systematically combines virtual reality/augmented reality (AR) remote surgical plan/rehearse hardware, customized 5G cloud computing and deep learning algorithms to provide real-time COVID-19 treatment scheme clues. Compared to existing perception therapy techniques, our new technique can significantly improve performance and security. The system collected 25 clinic data from the 347 positive and 2270 negative COVID-19 patients in the Red Zone by 5G transmission. After that, a novel auxiliary classifier generative adversarial network-based intelligent prediction algorithm is conducted to train the new COVID-19 prediction model. Furthermore, The Copycat network is employed for the model stealing and attack for the IoMT to improve the security performance. To simplify the user interface and achieve an excellent user experience, we combined the Red Zone's guiding images with the Green Zone's view through the AR navigate clue by using 5G. The XR surgical plan/rehearse framework is designed, including all COVID-19 surgical requisite details that were developed with a real-time response guaranteed. The accuracy, recall, F1-score, and area under the ROC curve (AUC) area of our new IoMT were 0.92, 0.98, 0.95, and 0.98, respectively, which outperforms the existing perception techniques with significantly higher accuracy performance. The model stealing also has excellent performance, with the AUC area of 0.90 in Copycat slightly lower than the original model. This study suggests a new framework in the COVID-19 diagnostic integration and opens the new research about the integration of XR and deep learning for IoMT implementation.

5G and beyond mobile systems target a plethora of emerging industrial and entertainment verticals that incur extra overhead to the network. These verticals are characterized by vigorous, continuous, and conflicting requirements that make the desired system's mission strenuous and more challenging. These verticals, such as autonomous driving, will accommodate immersive services, including virtual reality/augmented reality (VR/AR) and Holography services. Immersive services, in particular, have strict requirements for latency, throughput, and positioning. This article discusses VR-based remote services' potential, which occupies an important place among immersive services such as remote surgery, remote space control, and remote driving of Unmanned Aerial Vehicles (UAVs) or cars. Such services require an ultra-low Glass-to-Glass latency to avoid any failure and accidents when remotely controlling devices. We evaluate an immersive remote control service from the end-to-end communication perspectives using different camera devices that stream real-time 360° videos to a VR Head Mounted Device (HMD). The obtained results demonstrate the challenges of such service and the need for more advanced and optimized techniques, devices, and protocols to achieve less than 20 ms of Glass-to-Glass latency.

The ready availability of advanced visualization tools on picture archiving and communication systems workstations or even standard laptops through server-based or cloud-based solutions has enabled greater adoption of these techniques. We describe how radiologists can tailor imaging techniques for optimal 3D reconstructions provide a brief overview of the standard and newer "on-screen" techniques. We describe the process of creating 3D printed models for surgical simulation and education, with examples from the authors' institution and the existing literature. Finally, the review highlights current uses and potential future use cases for virtual reality and augmented reality applications in a pediatric neuroimaging setting.

The rapid development of virtual and augmented reality technologies is currently taking place in almost all spheres of activity. Elements of virtual and augmented reality are used in such areas as education, medicine, transport, gaming, tourism and others. The active spread of these technologies causes the emergence of special competencies in the IT labor market and, as a result, the formation of new professions.Many Russian universities are training students in IT training areas. Specialization in the development of computer games and virtual reality applications has begun recently. The provision of practical classes is accompanied by specific tasks, which gives students the opportunity to improve the use of software and technical devices.The relevance of the research is determined by the current demand for the use of the latest technologies by IT developers in the field of creating computer games. Today, technologies that provide a player’s immersion in virtual reality are becoming more and more popular. One of these technologies is a suit with wearable sensors that track a person’s position in space in real time. However, there are quite a few real described projects in the literature and on the Internet. This study examines the process of developing a task for creating a game application using virtual reality technology: a suit with wearable sensors for teaching students.Materials and methods of research. Timely identification of the needs of the IT market in personnel training allows educational organizations to form new training programs of different levels of training. This approach makes it possible to target the educational and methodological materials being developed to use the latest achievements in the development of the field under study.Using a systematic approach, the study characterizes virtual reality suits and sensors for monitoring the position in the user’s space. Thus, the goal of the task was to ensure the immersiveness and convenience of interaction between the player and the game environment.Based on materials on software, position sensors in space, the approach of pedagogical design was applied and the procedure was formed for a practical task, reflecting the relevant competencies.Results. The study was conducted on the basis in the framework of laboratory and practical work of students, as well as at a real enterprise. Training in the new profile of the direction of training “Applied informatics” is fully equipped with all the latest technologies in this field. As a result of the work, the content of the practical task was developed.Real development of virtual and augmented reality applications is conducted jointly with students. Almost all projects used a suit with body sensors.Conclusion. Our study examines in detail the process of developing an application using a suit with wearable sensors for further training of students. Based on the results, work can be carried out on real projects for any field. Based on the research materials, it is planned to issue a textbook for students with the profile of developing computer games and virtual / augmented reality applications.

The article presents an approach applied in the online training of pre-service teachers to acquire digital competence. The content of the concept of ‘digital competence’ in its sustainable and variable component is clarified. The understanding of ‘augmented reality’ to e-learning objects is presented. Two forms of ‘augmented reality’ have been proposed to visualize video information to a clarified concept. The first one is presented via a QR code for quick access and applicable for mobile learning. The other is provided by icons and is applicable to e-learning with a computer. It is believed that ‘augmented reality’ can differentiate students’ online learning according to the field-dependent and field-independent cognitive style and their preferences on the use of different digital learning devices.

The translator application from Banjar Language to Indonesian is called BandoAR and vice versa from Indonesian to Banjar Language which is called NdoBAR. Both applications utilize Mobile Augmented Reality technology with the Optical Character Recognition (OCR) Method for word recognition in the detected image. This application is recommended and used to help tourists visiting the city of Banjarmasin to be able to understand the language used by local people as a means of communication and for the general public who want to know the peculiarities of the Banjar language itself. The purpose of this study was to evaluate the user experience of the BandoAR and NdoBAR applications. The method used is UX Honeycomb, which has seven aspects to assess the user experience of an application. A total of 50 respondents were presented to assess the two applications. The results showed that the BandoAR application had an average UX Honeycomb value of 4.91 with a Very Strongly Agree predicate and for the NdoBAR application, an average value of UX Honeycomb was 4.89 with a Very Strongly Agree predicate. Both applications have fulfilled aspects of the user experience. However, we need some fixes for the shortcomings of both apps, to continue to improve interactions, better user experience, and other smart capabilities.

With the rapid development and extensive promotion of computer and Internet technology, users are more and more dependent on the network. The traditional information management system has been unable to meet the actual needs of a large number of concurrent access and fast storage. Therefore, it is very important to establish a unified cloud application platform for centralized management and unified identity of users in different systems. Through the combination of user experience testing technology, Internet of Things technology and cloud computing technology, the automatic user experience test of network application is realized, and the test report is generated. Managers can view the report through the report display interface provided by the user experience test cloud platform. Before the information system goes online, it will greatly improve the user experience quality of the information system by using the user experience test cloud platform. The designed pipeline integrates the augmented system and the Social Internet of Thing system to provide the efficient framework. Through the testing, the performance of the model is validated, the accuracy has reached the state-of-the-art level.

In this paper are discussed the application of 3D technologies, which are reflected in various areas of life, such as education, science, engineering and entertainment industry. The use of 3D technology in the educational process can significantly increase its efficiency by bringing the virtual computer environment closer to the real three-dimensional world. But for now the use of augmented reality in teaching is only gaining momentum. Many applications are still quite primitive, but developers are actively working on their refinement, increasing the amount of educational content, improving quality. The task of fast and high-quality algorithm for creating 3D content for further import in AR training projects has been urgent. The aim of this work is to increase the level of use of 3D technologies in the educational process for an effective and external interactive way of providing information. In the course of the work the methods of creating 3D content for different areas of application such as game industry, 3D visualization, 3D printing, AR projects, holographic fans, preliminary assessment of technical properties of the product and etc were considered. Data analysis has been shown that for each area of ​​application 3d content has its own requirements and features. This information was identified and presented in the form of a comparative table. Based on the analysis of modern methods of developing 3D content for various applications, a general method of creating 3D objects for further import into AR projects was proposed. Based on the proposed approach, a 3D model of the character for the children audio-encyclopedic stories educational application was designed. The obtained models were also integrated in the augmented reality environment. The use of this approach has increased the download speed of the model in an augmented reality application.

Augmented and Virtual Reality technology is a technology that continuesto develop every year, especially in the world of tourism and can be usedas an interactive promotional media. This study aims to see the use ofAugmented and Virtual Reality technology as an Interactive PromotionalMedia with qualitative methods. The results showed that there is theutilization of Augmented Reality technology with the name of DigitalMuseum with the help of Siji application and also the use of VirtualReality technology with the name of Online or Virtual Exhibition on thewebsite of the Museum of Proclamation Manuscript Formulation as aninteractive promotional media. The conclusion obtained from thisresearch shows that Siji application can be a means of interactivepromotional media and give a real impression while the OnlineExhibition can provide information and knowledge about the Museum ofProclamation Manuscript Formulation. It is recommended to continue toutilize the technology that develops every year.

This paper discussed the development of multi-maker augmented reality for learning Jawi in order to complement the formal study in school. We conducted an experiment with N=10 participants from Pusat Minda Lestari, UMS age 5 and 6 years old, to study the effectiveness of learning Jawi using the developed mobile augmented reality application. We prepared a test environment comprising an electroencephalography (EEG) system and mobile augmented reality (AR) application for analysis and testing. Results found that the learn ability of the students was improved after they used the mobile application to learn basic Jawi. The methodology used was ADDIE model, which included the analysis, design, development, implementation, and evaluate phases. This project is an innovation in learning Jawi and hopefully can increase the children’s interest in learning Jawi.

Currently, the obstacles faced in learning process are found to be low student interest in learning. The main factor contributing to low student interest in learning is influenced by the lack of variety of learning media used by teachers in schools. The use of augmented reality (AR) in education is classified as an important research topic, although it is still being studied in its early stages. The purpose of this research was to develop learning media using e-magazine based on an AR to increase students' interest in learning. The developed e-magazine was carried out using model of ADDIE (analysis, design, development, implementation and evaluation). Parameters in this study was focused on the quality of e-magazine and students’ interest in learning. Testing students' interest in learning was conducted by using a questionnaire and given directly to students. The experiments were carried out in the control class consisting of 36 students and the experimental class consisting of 36 students. Results of the study found that the development of AR-based e-magazines were obtained in relatively good category. This value was supported by the validation value found in the very valid category. Analysis on the didactic, constructive and technical aspects were obtained a score of 3,82, 3,85 and 3.78, respectively, with a very valid category. The results also showed that the use of AR-based e-magazines could be increased students' interest in learning and students' interest in learning was obtained at a value of 0.342 in the high category. Thus, the use of AR-based e-magazines as digital learning media can be used in the learning process.

In the modern era, preoperative planning is substantially facilitated by artificial reality technologies, which permit a better understanding of patient anatomy, thus increasing the safety and accuracy of surgical interventions. In the field of orthopedic surgery, the increase in safety and accuracy improves treatment quality and orthopedic patient outcomes. Artificial reality technologies, which include virtual reality (VR), augmented reality (AR), and mixed reality (MR), use digital images obtained from computed tomography or magnetic resonance imaging. VR replaces the user's physical environment with one that is computer generated. AR and MR have been defined as technologies that permit the fusing of the physical with the virtual environment, enabling the user to interact with both physical and virtual objects. MR has been defined as a technology that, in contrast to AR, enables users to visualize the depth and perspective of the virtual models. We aimed to shed light on the role that MR can play in the visualization of orthopedic surgical anatomy. The literature suggests that MR could be a valuable tool in orthopedic surgeon's hands for visualization of the anatomy. However, we remark that confusion exists in the literature concerning the characteristics of MR. Thus, a more clear description of MR is needed in orthopedic research, so that the potential of this technology can be more deeply understood.

Critique sessions are an essential educational activity at the center of many design disciplines, especially those involving the creation of physical mockups. Conventional approaches often require the students and the instructor to be in the same space to jointly view and discuss physical artifacts. However, in remote learning contexts, available tools (such as videoconferencing) are insufficient due to ineffective, inefficient spatial referencing. This paper presents ARCritique, a mobile Augmented Reality application that allows users to 1) scan physical artifacts, generate corresponding 3D models, and share them with distant instructors; 2) view the model simultaneously in a synchronized virtual environment with remote collaborators; and 3) point to and draw on the model synchronously to aid communication. We evaluated ARCritique with seven Industrial Design students and three faculty to use the app in a remote critique setting. The results suggest that direct support for spatial communication improves collaborative experiences. Published version

Pengetahuan mengenai perlengkapan kemiliteran tidak hanya diperoleh melalui pameran saja, dengan adanya perkembangan teknologi informasi bisa diperoleh melalui aplikasi. Penggunaan aplikasi augmented reality dapat dijadikan contoh dalam pengetahuan perlengkapan militer secara tiga dimensi. Perancangan ini bertujuan memberikan pengetahuan kepada masyarakat umum dengan menggunakan teknologi Augmented Reality. Perancangan ini menggunakan metode ADDIE dan algoritma FAST Corner dan Lucas Kanade. Perancangan ini menghasilkan sebuah aplikasi yang berisi pengenalan perlengkapan militer yang terdiri dari baret, topi boonie, topi patroli, helm, kacamata, tas besar, P3K (Pertolongan Pertama Pada kecelakaan), pisau, senjata jauh dan senjata menengah. Informasi singkat mengenai masing-masing perlengkapan militer yang dirancang secara menarik agar masyarakat umum mudah memahami dalam menggunakan aplikasi yang dapat dijalankan pada platform Android.

The objective of the present study is to produce a learning media product of health protocol implementation to children entitled "Awas Ada Virus Corona". This media utilized RD method and took several steps, namely preliminary research and data collection, research planning, product development, expert validation, product revision and final product. Feasibility testing is carried out by media experts by examining the multimedia construction domain consist of 3 aspects, namely guidance and information, program performance as well as systematic, aesthetic, and design aspects. Furthermore, a filled questionnaire by media experts was used to collect the data needed. Meanwhile, the descriptive statistic was used to analyzed the data gathered. The result of the present study is an application that can be used by children entitled " Awas Ada Virus Corona " as an educational media in implementing 3M of health protocol, namely wearing masks, washing hands, and maintaining a safe distance. The addition of Augmented reality technology with a live coloring feature is added in this application in reading markers to increase children's attractiveness. This augmented reality technology is applied in order to avoid the children only focus on gadgets, so children feel happy in using the familiar media and still interacting with the environment.

Penelitian ini bertujuan untuk menghasilkan media pembelajaran lingkaran dengan augmented reality berbasis Android yang valid dan praktis bagi siswa tunarungu. Media pembelajaran tersebut akan dinilai kelayakannya dari segi kevalidan dan kepraktisan. Penelitian ini termasuk penelitian pengembangan (R&D) dengan menggunakan model ADDIE yang terdiri atas empat tahap yaitu analisis, pendesainan, pengembangan, implementasi dan tahap evaluasi. Subjek dalam penelitian ini adalah 17 siswa tunarungu kelas VII, VIII dan IX di SLBN 2 Bantul. Instrumen penelitian ini terdiri dari instrumen angket penilaian kevalidan dari segi materi dan media, serta lembar angket respon siswa. Analisis data skor validasi ahli dan skor respon siswa yang berbentuk skor skala lima (kategori kuantitatif), dikonversi ke dalam kategori kualitatif. Hasil penelitian berupa media lingkaran dengan augmented reality berbasis Android yang valid dan praktis. Ditinjau dari hasil penilaian ahli materi dan ahli media, kevalidan media termasuk kategori sangat baik. Sedangkan dari penilaian respon siswa, kepraktisan media termasuk kategori sangat baik. Siswa tunarungu terlihat antusias menggunakan media pembelajaran yang mengakomodir keterbatasan indra pendengarannya, melalui penyajian teks sebagai pengganti audio dalam mengilustrasikan konten visual konsep lingkaran. Meskipun sudah memenuhi kriteria kevalidan dan kepraktisan, namun media yang dikembangkan masih memiliki keterbatasan yang memerlukan adanya penelitian lebih lanjut menurut penilaian guru.

The work is devoted to the description of the process of developing a universal toolkit for generating synthetic data for training various neural networks. The approach used has shown its success and effectiveness in solving various problems, in particular, training a neural network to recognize shopping behavior inside stores through surveillance cameras and training a neural network for recognizing spaces with augmented reality devices without using auxiliary infrared cameras. Generalizing conclusions allow planning the further development of technologies for generating three-dimensional synthetic data.

The use of technology in special education has been rapidly increased in recent years. Augmented reality (AR) is one of these technologies that combines real world and virtual objects together. AR ...

With the rapid development of information technology, mobile in recent years, with the rapid development of information technology, mobile 5G, virtual reality, augmented reality, big data, cloud co...

Augmented Reality is a type of interactive technology that combines real objects and virtual objects at the same time and is displayed on the screen. Augmented Reality has been applied as a way of working based on image detection and it is commonly called a marker. By using the smartphone's camera and then detects the marker that has been printed. Augmented Reality is widely used in various purposes, the example is for advertising or promotion. In advertising, Augmented Reality can be used as a promotion or information that can make it more interesting. This research aims to apply AR technology to the campus promotion media on the University of Panca Marga's new student admission brochure. The stages taken in this research are literature study as well as observation, design, and implementation stages. The application of Augmented Reality was built using Unity 3D software, Android SDK, Vuforia SDK, and Sweet Home 3D. The results of the application of Augmented Reality technology can display 3D virtual objects, 2D, and video using the new student admission brochure used as a marker. The marker that is detected by the camera on the Android smartphone can display 3D UPM buildings, location plans, UPM profile videos, and faculty digital brochures. So users can see how the campus buildings are in real-time as well as some other information that has been provided. Based on the tests, all features can run well.

Marc-doa application is Mobile Augmented Reality cubes used as a learning media for children, focusing on cognitive and psychomotor learning. The contents of this application are daily prayers for Muslim children and designed in the form of audio-video animations. The development of the Marc-doa application uses the Multimedia Development Life Cycle method. This research aims to discuss the design and evaluation of Marc-doa. Evaluation and technical tests are conducted on several aspects, specifically on the detection process, distance, angle, and surface area. In addition, the testing process is to identify the ideal conditions for the use of Marc-doa. From the evaluation and testing process, four markers showed good results. Only one marker does not show the results as expected. The ideal distance for using Marc-doa application between 25-40 cm. The angle for ideal reception between 0-30 degrees and the surface area Marc-doa application can detect markers correctly between 50%-100% surface area. The ideal distance, surface area, and conditions for utilising Marc-doa application were obtained from this research.

Novice users of telesurgery could be limited by their experience and technical ability. The impact of the COVID-19 pandemic on health care systems is unprecedented, and telehealth allowed care providers and patients a safety margin. An indirect impact of redeployment of hospital staff during COVID-19 management has been on the reduced educational opportunities for residents. Proximie can be considered as a virtual teaching platform or classroom for any user. Twenty-one students voluntarily participated in utilizing a da Vinci® skills simulator (dVSS) to carry out surgical training simulation tasks. Our study focuses on digital native’s adaptation to utilizing Proximie’s augmented reality platform to direct task performance, to gauge its feasibility by this unique cohort.

In terms of the education system, developing countries of the world are still characterized with unequal school systems due to perpetuating inequality, and poverty. It is said that children who are admitted into the top schools are bound to achieve good grades in mathematics and sciences, unlike children in the next 6600 schools combined. This is a result of the majority of learners not having access to proper educational facilities, such as scientific labs and other technological means that would be in favor of the learner, unlike the rest of their counterparts in private or privileged schools. This paper presents a virtual reality science application, specifically focusing on the concept of inertia, which was achieved using various techniques such as 3D visualization and computer graphics to provide a platform for training and education of the scientific concept. Studies show that with the use of this technology, users were able to improve their subject scores, and it assisted with memory retention as well. With just the aid of a cell phone and virtual reality glasses, the ordinary learner will now be able to navigate his way around certain scientific concepts and experiments. Furthermore, the learner will be able to comprehend the scientific concept that is presented in front of their eyes, and this will ensure that every learner has equitable access to the learning content.

The rise of online learning poses unique challenges in music education, where live demonstration and musical synchronization are critical for student success. We present HoloKeys, a music education interface which allows instructors to play remotely located pianos using an augmented reality headset and wifi-enabled microcontrollers. This approach allows students to receive distance education which is more direct, immersive, and comprehensive than conventional video conferencing allows for. HoloKeys enables remote students to observe live instructional demonstration on a physical keyboard in their immediate environment just as they would in traditional settings. HoloKeys consists of two separate components: an augmented reality user interface and a piano playing apparatus. Our system aims to extend online music education beyond desktop platforms into the physical world, thereby addressing crucial obstacles encountered by educators and students transitioning into online education.

In this work, we present a system that adds augmented reality interaction and 3D-space utilization to educational videoconferencing for a more engaging distance learning experience. We developed infrastructure and user interfaces that enable the use of an instructor’s physical 3D space as a teaching stage, promote student interaction, and take advantage of the flexibility of adding virtual content to the physical world. The system is implemented using hand-held mobile augmented reality to maximize device availability, scalability, and ready deployment, elevating traditional video lectures to immersive mixed reality experiences. We use multiple devices on the teacher’s end to provide different simultaneous views of a teaching space towards a better understanding of the 3D space.

In-stadium sports spectating delivers a unique social experience in a variety of sports. However, in contrast to broadcast delivery, it lacks the provision of real-time information augmentation, like game statistics overlaid on screen. In an earlier iteration, we developed ARSpectator, a prototypical, mobile system which can be brought to the stadium to experience both, the live sport action and situated infographics spatially augmented into the scene. In some situations it is difficult or often impossible to go to the stadium though, for instance because of limited stadium access during pandemics or when wanting to conduct controlled user studies. We address this by turning our ARSpectator system into an indirect augmented reality experience deployed to an immersive, virtual reality head-mounted display: The live stadium experience is delivered by way of a surrounding 360 video recording while maintaining and extending the provision of interactive, situated infographics. With our XRSpectator demo prototype presented here, users can have an ARSpectator experience of a rugby game in our local stadium.

Holographic displays enable the formation of a three-dimensional particle field distribution and is promised to be one solution of the key problems in present virtual and augmented reality displays. The biggest challenge in the computer-generated holography (CGH) is the high consuming calculation time and the restricted field of view due to the micro display screen. In order to solve the above mentioned problems, we propose a method for machine learning accelerated lensless holographic projection, which is based on the reconstruction of complex amplitude image from phase-only CGH. The runtime of generating the phase-only CGH is half of traditional iterative method. The complex amplitude image can achieve the maximum diffraction bandwidth of the spatial light modulator (SLM). The feasibility of this method is verified by experiments.

The goal of the WrightBroS project is to design a system supporting the training of pilots in a flight simulator. The desired software should work on smart glasses supplementing the visual information with augmented reality data, displaying, for instance, additional training information or descriptions of visible devices in real time. Therefore, the rapid recognition of observed objects and their exact positioning is crucial for successful deployment. The keypoint descriptor approach is a natural framework that is used for this purpose. For this to be applied, the thorough examination of specific keypoint location methods and types of keypoint descriptors is required first, as these are essential factors that affect the overall accuracy of the approach. In the presented research, we prepared a dedicated database presenting 27 various devices of flight simulator. Then, we used it to compare existing state-of-the-art techniques and verify their applicability. We investigated the time necessary for the computation of a keypoint position, the time needed for the preparation of a descriptor, and the classification accuracy of the considered approaches. In total, we compared the outcomes of 12 keypoint location methods and 10 keypoint descriptors. The best scores recorded for our database were almost 96% for a combination of the ORB method for keypoint localization followed by the BRISK approach as a descriptor.

At the core of augmented reality audio (ARA) technology lies the ARA mix, a process responsible for the assignment of a virtual environment to a real one. Legacy ARA mix models have focused on the natural reproduction of the real environment, whereas the virtual environment is simply mixed through fixed gain methods. This study presents a novel approach of a dynamic ARA mix that facilitates a smooth adaptation of the virtual environment to the real one, as well as dynamic control of the virtual audio engine, by taking into account the inherent characteristics of both ARA technology and binaural auditory perception. A prototype feature extraction technique of auditory perception characteristics through a real-time binaural loudness prediction method was used to upgrade the legacy ARA mix model into a dynamic model, which was evaluated through benchmarks and subjective tests and showed encouraging results in terms of functionality and acceptance.

Due to the evolution of motion capture devices, natural user interfaces have been applied in several areas, such as neuromotor rehabilitation supported by virtual environments. This paper presents a smartphone application that allows the user to interact with the virtual environment and enables the captured data to be stored, processed, and used in machine learning models. The application submits the recordings to the remote database with information about the movement and in order to apply supervised machine learning. As a proof of concept, we generated a dataset capturing movement data using our application with 232 instances divided into 8 classes of movements. Moreover, we used this dataset for training models that classifies these movements. The remarkable accuracy of the models shows the feasibility of using body articulation data for a classification task after some data transformations.

A traditional house is a house that has a unique building characteristic in an area that describes the culture of the area itself. Traditional houses are also unique in terms of form, function and meaning of the building. As Indonesians, we must introduce one of the existing cultures so that it is not forgotten by the community, especially school children. Judging from the development of technology, we must make new things in introducing traditional houses. From the results of research conducted at SD IT BINA INSANI, the teachers there still use conventional media in the form of textbooks and thematic books for the learning process. Therefore, the authors created an application to introduce western Indonesian traditional houses using Augmented Reality called RA-IBB as an introduction medium. This application is based on Android so that teachers can easily explain material about traditional houses and can show traditional houses in 3D without having to come directly to the province. And for students to be able to study traditional houses wherever and whenever. The making of this application uses the waterfall method for the development stage. The RA-IBB application contains materials about western Indonesian traditional houses and their 3d objects and quiz questions that use random shuffle algorithm questions. The random shuffle algorithm is a randomized question from an array or record. Based on the User Acceptance Test (UAT) for teachers, the results obtained from several aspects, namely the design aspect yields a percentage of 91%, the application information aspect produces a percentage of 91.

Urban information discovery has a long history in geography and other related disciplines. Routing and Point-of-Interest (POI) searching services have been implemented in many Geographic Information Systems (GISs)-based applications, where spatial information (e.g., POI data) is projected to a digital map as a source of auxiliary data to support intelligent spatial decision-making. In recent years, Augmented Reality (AR) has become a growing trend to visualize spatial objects through digital visual elements. However, a research gap exists between GIS and AR systems in terms of computational efficiency and interoperability. This paper aims to develop a geospatial cyberinfrastructure that integrates urban information and AR to enhance spatial knowledge visualization and discovery. Our experiments demonstrate how spatial analysis and urban sentiment computing can efficiently be integrated into the AR-based framework to support real-time decision-making.

Virtual Humans are becoming a commodity in computing technology and lately have beenutilized in the context of interactive presentations in Virtual Cultural Heritage environments and exhibitions. To this end, this research work underlines the importance of aligning and fine-tuning Virtual Humans’ appearance to their roles and highlights the importance of affective components. Building realistic Virtual Humans was traditionally a great challenge requiring a professional motion capturing studio and heavy resources in 3D animation and design. In this paper, a workflow for their implementation is presented, based on current technological trends in wearable mocap systems and advancements in software technology for their implementation, animation, and visualization. The workflow starts from motion recording and segmentation to avatar implementation, retargeting, animation, lip synchronization, face morphing, and integration to a virtual or physical environment. The testing of the workflow occurs in a use case for the Mastic Museum of Chios and the implementation is validated both in a 3D virtual environment accessed through Virtual Reality and on-site at the museum through an Augmented Reality application. The findings, support the initial hypothesis through a formative evaluation, and lessons learned are transformed into a set of guidelines to support the replication of this work.

The University of Glasgow accommodates a large number of new students every year. Arriving at a new campus can seem daunting at first, as it is always a challenge to navigate a university’s many buildings and pathways. Although in-person and self-guided online campus tours are currently offered by the University, in-person tours require advance booking and online tours do not provide the same level of immersion. As a solution, this paper describes the design, development and evaluation of an alternative: a location-based game for campus exploration. The prototype was evaluated remotely by eight participants in two stages, revealing technical issues that were subsequently addressed. A final, on-campus evaluation was carried out by a single participant using the final version of the game, with positive results. By utilizing player location tracking, this game successfully integrates a classic murder-mystery style story into a real-life setting–the University’s campus–to provide an alternative means for new students to become familiar with their surroundings .

Regardless of the rapid developments in the Internet of Things (IoT) world, its integration into our daily lives has been trailing slowly. We believe that the issue stems from vendor-specific products, restricting customers to their proprietary software and limiting interoperability across vendors. In our earlier research, we proposed a Mixed Reality Control Panel (MRCP) framework to tackle this gap. By offering a vendor-neutral, open-source, cross-platform, mixed reality (MR) software allowing users to seamlessly interact with headless IoT devices. The MRCP application, Smart Parking Meter, and Smart Lock were developed within this framework, as proofs of concepts. In this work, the proposed system’s efficiency to perform each action is compared against existing solutions, as well as the benefits of additional smart features. Showcasing how an MR interface allows for quick interaction with IoT devices. Early development decisions on being cross-platform and vendor-neutral allow for future scalability. Development took place in the networking aspect by implementing a smart parking solution for the parking meter. As well as strengthening its security by performing an attack against the system and developing a countermeasure to patch it. Concluding that both devices under the MRCP system are viable solutions for future smart city integration.

While the new pneumonia epidemic (CVD-19) has had an unprecedented impact on the traditional economic model, it has also given a huge boost to the new retail industry immersive technologies such as Virtual Reality (VR) and Augmented Reality (AR). Mixed Reality (MR)will achieve breakthrough development of even more 5G technology, focusing on digital media with multi-sensory interaction features such as visual, auditory, and kinesthetic-immersive media-to build a new paradigm for commercial information dissemination.Therefore, it is of obvious relevance to understand the value of immersive media in the digital retail space an to find the right way to apply it to the way coastal cities grow economically.

Augmented reality applications running on smartphones or tablets are becoming increasingly common. It is crucial to extract the physical structure of the scene perceived by the device camera in these applications. In such applications, employed in education, navigation and obstacle notification, the distance information between the device camera and the object must be derived and processed with sufficient accuracy and speed. In this study, the deep learning model, named "From Big To Small (BTS)", with superior performance metrics in depth extraction according to the literature reviews, was transformed into a web service and tested on an Android phone. Thus, a deep learning model with a high computational cost will be available on an Android device with average processing power. Test results were examined, and improvements were discussed.

Human speech often contains ambiguity stemming from the use of demonstrative pronouns (DPs), such as “this” and “these.” While we can typically decipher which objects of interest DPs are referring to based on context, modern day voice assistants (VAs – such as Google Assistant and Siri) are yet unable to process queries containing such ambiguity. For instance, to humans, a question such as “how much is this?” can be clarified through visual reference (e.g., a buyer gestures to the seller the object they would like to purchase). To bridge this gap between human and machine cognition, we built and examined a touch + voice multimodal VA prototype that enables users to select key spatial information to embed as context and query the VA. The prototype converts results of mobile, real-time object recognition and optical character recognition models into augmented reality buttons that represent features. Users can interact with and modify the selected features through a word grid. We conducted a study to investigate: 1) how touch performs as an additional modality to resolve ambiguity in queries, 2) how users use DPs when interacting with VAs, and 3) how users perceive a VA that can understand DPs. From this procedure we found that as the query becomes more complex, users prefer the multimodal VA over the standard VA without experiencing elevated cognitive load. Additionally, even though it took some time getting used to, many participants eventually became comfortable with using DPs to interact with the multimodal VA and appreciated the improved human-likeness of human-VA conversations.

Reconstructing point clouds from images would extremely benefit many practical CV applications, such as robotics, automated vehicles, and Augmented Reality. Fueled by the advances of deep neural network, many deep learning frameworks are proposed to address this problem recently. However, these frameworks generally rely on a large amount of labeled training data (e.g., image and point cloud pairs). Although we usually have numerous 2D images, corresponding 3D shapes are insufficient in practice. In addition, most available 3D data covers only a limited amount of classes, which further restricts the models' generalization ability to novel classes. To mitigate these issues, we propose a novel few-shot single-view point cloud generation framework by considering both class-specific and class-agnostic 3D shape priors. Specifically, we abstract each class by a prototype vector that embeds class-specific shape priors. Class-agnostic shape priors are modeled by a set of learnable shape primitives that encode universal 3D shape information shared across classes. Later, we combine the input image with class-specific prototypes and class-agnostic shape primitives to guide the point cloud generation process. Experiments on the popular ModelNet and ShapeNet datasets demonstrate that our method outperforms state-of-the-art methods in the few-shot setting.

Internet of Things (IoT), Cloud and Augmented Reality (AR) are the emerging and developing technologies and are at the horizon and hype of their life cycle. Lots of commercial applications based on IoT, cloud and AR provide unrestricted access to data. The real-time applications based on these technologies are at the cusp of their innovations. The most frequent security attacks for IoT, cloud and AR applications are DDoS attacks. In this paper a detailed account of various DDoS attacks that can be the hindrance of many important sensitive services and can degrade the overall performance of recent services which are purely based on network communications. The DDoS attacks should be dealt with carefully and a set of a new generations of algorithm need to be developed to mitigate the problems caused by non-repudiation kinds of attacks.

The field of Augmented Reality (AR) has seen a rapid expansion within the past decades, with many sectors nowadays applying AR methodologies for their own particular visualization needs. One field in particular that has adopted AR visualization techniques is the geospatial sciences, as they have identified the inherent spatial nature of AR as particularly applicable for visualizing (geo)spatial relationships in systems of interest. However, while we can find multiple classification schemes for AR applications in AR literature, to our knowledge no such classification exists that focuses specifically on the spatial aspects of AR applications, namely the visualization size, scale, and acknowledgement of and alignment with the user’s physical environment. Therefore no AR classification exists that is specifically applicable to the geospatial sciences. In this paper we present an initial classification of AR approaches for spatial data visualization, highlighting the different spatial characteristics that are of particular relevance to the geospatial sciences. We expect that the initial classification presented here will help researchers working with AR visualizations in organizing their application design with regard to its spatial characteristics.