Search

Searching for objects is a common task in daily life and work. For augmented reality (AR) devices without spatial perception systems, the image of the object's last appearance serves as a common search assistance. Compared to using only images as visual cues, videos capturing the process of object placement can provide procedural guidance, potentially enhancing users' search efficiency. However, complete video playback capturing the entire object placement process as visual cues can be excessively lengthy, requiring users to invest significant viewing time. To explore whether segmented or accelerated video playback can still assist users in object retrieval tasks effectively, we conducted a user study. The results indicated that when video playback is covering the first appearance of the object's destination to the object's final appearance (referred to as the destination appearance, DA) and playing at normal speed, search time and cognitive load were significantly reduced. Subsequently, we designed a second user study to evaluate the performance of video playback compared to image cues in object retrieval tasks. The results showed that combining the DA playback starting point with images of the object's last appearance further reduced search time and cognitive load. [ABSTRACT FROM AUTHOR]

In recent decades, technological advancements have transformed the industry, highlighting the efficiency of automation and safety. The integration of augmented reality (AR) and gesture recognition has emerged as an innovative approach to create interactive environments for industrial equipment. Gesture recognition enhances AR applications by allowing intuitive interactions. This study presents a web-based architecture for the integration of AR and gesture recognition, designed to interact with industrial equipment. Emphasizing hardware-agnostic compatibility, the proposed structure offers an intuitive interaction with equipment control systems through natural gestures. Experimental validation, conducted using Google Glass, demonstrated the practical viability and potential of this approach in industrial operations. The development focused on optimizing the system's software and implementing techniques such as normalization, clamping, conversion, and filtering to achieve accurate and reliable gesture recognition under different usage conditions. The proposed approach promotes safer and more efficient industrial operations, contributing to research in AR and gesture recognition. Future work will include improving the gesture recognition accuracy, exploring alternative gestures, and expanding the platform integration to improve the user experience. [ABSTRACT FROM AUTHOR]

An educational augmented reality auscultation system (EARS) is proposed to enhance the reality of auscultation training using a simulated patient. The conventional EARS cannot accurately reproduce breath sounds according to the breathing of a simulated patient because the system instructs the breathing rhythm. In this study, we propose breath measurement methods that can be integrated into the chest piece of a stethoscope. We investigate methods using the thoracic variations and frequency characteristics of breath sounds. An accelerometer, a magnetic sensor, a gyro sensor, a pressure sensor, and a microphone were selected as the sensors. For measurement with the magnetic sensor, we proposed a method by detecting the breathing waveform in terms of changes in the magnetic field accompanying the surface deformation of the stethoscope based on thoracic variations using a magnet. During breath sound measurement, the frequency spectra of the breath sounds acquired by the built-in microphone were calculated. The breathing waveforms were obtained from the difference in characteristics between the breath sounds during exhalation and inhalation. The result showed the average value of the correlation coefficient with the reference value reached 0.45, indicating the effectiveness of this method as a breath measurement method. And the evaluations suggest more accurate breathing waveforms can be obtained by selecting the measurement method according to breathing method and measurement point. [ABSTRACT FROM AUTHOR]

In the era of aging civil infrastructure and growing concerns about rapid structural deterioration due to climate change, the demand for real-time structural health monitoring (SHM) techniques has been predominant worldwide. Traditional SHM methods face challenges, including delays in processing acquired data from large structures, time-intensive dense instrumentation, and visualization of real-time structural information. To address these issues, this paper develops a novel real-time visualization method using Augmented Reality (AR) to enhance vibration-based onsite structural inspections. The proposed approach presents a visualization system designed for real-time fieldwork, enabling detailed multi-sensor analyses within the immersive environment of AR. Leveraging the remote connectivity of the AR device, real-time communication is established with an external database and Python library through a web server, expanding the analytical capabilities of data acquisition, and data processing, such as modal identification, and the resulting visualization of SHM information. The proposed system allows live visualization of time-domain, frequency-domain, and system identification information through AR. This paper provides an overview of the proposed technology and presents the results of a lab-scale experimental model. It is concluded that the proposed approach yields accurate processing of real-time data and visualization of system identification information by highlighting its potential to enhance efficiency and safety in SHM by integrating AR technology with real-world fieldwork. [ABSTRACT FROM AUTHOR]

Outdoor Location-Based Augmented Reality (LAR) applications require precise positioning for seamless integrations of virtual content into immersive experiences. However, common solutions in outdoor LAR applications rely on traditional smartphone sensor fusion methods, such as the Global Positioning System (GPS) and compasses, which often lack the accuracy needed for precise AR content alignments. In this paper, we introduce an innovative approach to enhance LAR anchor precision in outdoor environments. We leveraged Visual Simultaneous Localization and Mapping (VSLAM) technology, in combination with innovative cloud-based methodologies, and harnessed the extensive visual reference database of Google Street View (GSV), to address the accuracy limitation problems. For the evaluation, 10 Point of Interest (POI) locations were used as anchor point coordinates in the experiments. We compared the accuracies between our approach and the common sensor fusion LAR solution comprehensively involving accuracy benchmarking and running load performance testing. The results demonstrate substantial enhancements in overall positioning accuracies compared to conventional GPS-based approaches for aligning AR anchor content in the real world. [ABSTRACT FROM AUTHOR]

(1) Background: A current trend observed in the logistics sector is the use of Industry 4.0 tools to improve and enhance the efficiency of cargo handling processes. One of the popular solutions is an augmented reality system that supports operators in everyday tasks. The article aims to present design assumptions for implementing an augmented reality system to support air cargo handling at the warehouse. (2) Methods: Research was carried out based on a five-stage analytical procedure, aiming to analyze the current state and identify the potential for implementing the AR system. The following methods were used to collect data: co-participant observations, process analysis, direct interviews, analysis of internal documentation, and applicable legal regulations. (3) Results: The conducted research allowed for identifying information flows accompanying cargo flows and developing a project to automate selected information flows. The obtained results made it possible to identify operations for which the AR system's implementation will increase their effectiveness and efficiency. (4) Conclusions: The obtained results identified the need to develop a hybrid algorithm for arranging cargo in the warehouse and to build a system supporting self-verification of markings on air cargo. [ABSTRACT FROM AUTHOR]

The industry is currently undergoing a digital revolution driven by the integration of several enabling technologies. These include automation, robotics, cloud computing, industrial cybersecurity, systems integration, digital twins, etc. Of particular note is the increasing use of digital twins, which offer significant added value by providing realistic and fully functional process simulations. This paper proposes an approach for developing digital twins in industrial environments. The novelty lies in not only focusing on obtaining the model of the industrial system and integrating virtual reality and/or augmented reality but also in emphasizing the importance of incorporating other enabled technologies of Industry 4.0, such as system integration, connectivity with standard and specific industrial protocols, cloud services, or new industrial automation systems, to enhance the capabilities of the digital twin. Furthermore, a proposal of the software tools that can be used to achieve this incorporation is made. Unity is chosen as the real-time 3D development tool for its cross-platform capability and streamlined industrial system modeling. The integration of augmented reality is facilitated by the Vuforia SDK. Node-RED is selected as the system integration option, and communications are carried out with MQTT protocol. Finally, cloud-based services are recommended for effective data storage and processing. Furthermore, this approach has been used to develop a digital twin of a robotic electro-pneumatic cell. [ABSTRACT FROM AUTHOR]

Addressing conventional neurosurgical navigation systems' high costs and complexity, this study explores the feasibility and accuracy of a simplified, cost-effective mixed reality navigation (MRN) system based on a laser crosshair simulator (LCS). A new automatic registration method was developed, featuring coplanar laser emitters and a recognizable target pattern. The workflow was integrated into Microsoft's HoloLens-2 for practical application. The study assessed the system's precision by utilizing life-sized 3D-printed head phantoms based on computed tomography (CT) or magnetic resonance imaging (MRI) data from 19 patients (female/male: 7/12, average age: 54.4 ± 18.5 years) with intracranial lesions. Six to seven CT/MRI-visible scalp markers were used as reference points per case. The LCS-MRN's accuracy was evaluated through landmark-based and lesion-based analyses, using metrics such as target registration error (TRE) and Dice similarity coefficient (DSC). The system demonstrated immersive capabilities for observing intracranial structures across all cases. Analysis of 124 landmarks showed a TRE of 3.0 ± 0.5 mm, consistent across various surgical positions. The DSC of 0.83 ± 0.12 correlated significantly with lesion volume (Spearman rho = 0.813, p < 0.001). Therefore, the LCS-MRN system is a viable tool for neurosurgical planning, highlighting its low user dependency, cost-efficiency, and accuracy, with prospects for future clinical application enhancements. [ABSTRACT FROM AUTHOR]

Visual localization refers to the process of determining an observer's pose by analyzing the spatial relationships between a query image and a pre-existing set of images. In this procedure, matched visual features between images are identified and utilized for pose estimation; consequently, the accuracy of the estimation heavily relies on the precision of feature matching. Incorrect feature matchings, such as those between different objects and/or different points within an object in an image, should thus be avoided. In this paper, our initial evaluation focused on gauging the reliability of each object class within image datasets concerning pose estimation accuracy. This assessment revealed the building class to be reliable, while humans exhibited unreliability across diverse locations. The subsequent study delved deeper into the degradation of pose estimation accuracy by artificially increasing the proportion of the unreliable object—humans. The findings revealed a noteworthy decline started when the average proportion of the humans in the images exceeded 20%. We discuss the results and implications for dataset construction for visual localization. [ABSTRACT FROM AUTHOR]

Recently, realistic services like virtual reality and augmented reality have gained popularity. These realistic services require deterministic transmission with end-to-end low latency and high reliability for practical applications. However, for these real-time services to be deterministic, the network core should provide the requisite level of network. To deliver differentiated services to each real-time service, network service providers can classify applications based on traffic. However, due to the presence of personal information in headers, application classification based on encrypted application data is necessary. Initially, we collected application traffic from four well-known applications and preprocessed this data to extract encrypted application data and convert it into model input. We proposed a lightweight transformer model consisting of an encoder, a global average pooling layer, and a dense layer to categorize applications based on the encrypted payload in a packet. To enhance the performance of the proposed model, we determined hyperparameters using several performance evaluations. We evaluated performance with 1D-CNN and ET-BERT. The proposed transformer model demonstrated good performance in the performance evaluation, with a classification accuracy and F1 score of 96% and 95%, respectively. The time complexity of the proposed transformer model was higher than that of 1D-CNN but performed better in application classification. The proposed transformer model had lower time complexity and higher classification performance than ET-BERT. [ABSTRACT FROM AUTHOR]

Accurate prediction of the estrus period is crucial for optimizing insemination efficiency and reducing costs in animal husbandry, a vital sector for global food production. Precise estrus period determination is essential to avoid economic losses, such as milk production reductions, delayed calf births, and disqualification from government support. The proposed method integrates estrus period detection with cow identification using augmented reality (AR). It initiates deep learning-based mounting detection, followed by identifying the mounting region of interest (ROI) using YOLOv5. The ROI is then cropped with padding, and cow ID detection is executed using YOLOv5 on the cropped ROI. The system subsequently records the identified cow IDs. The proposed system accurately detects mounting behavior with 99% accuracy, identifies the ROI where mounting occurs with 98% accuracy, and detects the mounting couple with 94% accuracy. The high success of all operations with the proposed system demonstrates its potential contribution to AR and artificial intelligence applications in livestock farming. [ABSTRACT FROM AUTHOR]

Camera pose estimation is vital in fields like robotics, medical imaging, and augmented reality. Fiducial markers, specifically ArUco and Apriltag, are preferred for their efficiency. However, their accuracy and viewing angle are limited when used as single markers. Custom fiducial objects have been developed to address these limitations by attaching markers to 3D objects, enhancing visibility from multiple viewpoints and improving precision. Existing methods mainly use square markers on non-square object faces, leading to inefficient space use. This paper introduces a novel approach for creating fiducial objects with custom-shaped markers that optimize face coverage, enhancing space utilization and marker detectability at greater distances. Furthermore, we present a technique for the precise configuration estimation of these objects using multiviewpoint images. We provide the research community with our code, tutorials, and an application to facilitate the building and calibration of these objects. Our empirical analysis assesses the effectiveness of various fiducial objects for pose estimation across different conditions, such as noise levels, blur, and scale variations. The results suggest that our customized markers significantly outperform traditional square markers, marking a positive advancement in fiducial marker-based pose estimation methods. [ABSTRACT FROM AUTHOR]

Eye gaze can be a potentially fast and ergonomic method for target selection in augmented reality (AR). However, the eye-tracking accuracy of current consumer-level AR systems is limited. While state-of-the-art AR target selection techniques based on eye gaze and touch (gaze-touch), which follow the "eye gaze pre-selects, touch refines and confirms" mechanism, can significantly enhance selection accuracy, their selection speeds are usually compromised. To balance accuracy and speed in gaze-touch grid menu selection in AR, we propose the Hand-Held Sub-Menu (HHSM) technique.tou HHSM divides a grid menu into several sub-menus and maps the sub-menu pointed to by eye gaze onto the touchscreen of a hand-held device. To select a target item, the user first selects the sub-menu containing it via eye gaze and then confirms the selection on the touchscreen via a single touch action. We derived the HHSM technique's design space and investigated it through a series of empirical studies. Through an empirical study involving 24 participants recruited from a local university, we found that HHSM can effectively balance accuracy and speed in gaze-touch grid menu selection in AR. The error rate was approximately 2%, and the completion time per selection was around 0.93 s when participants used two thumbs to interact with the touchscreen, and approximately 1.1 s when they used only one finger. [ABSTRACT FROM AUTHOR]

In this paper, we introduce LagunAR, a mobile outdoor Augmented Reality (AR) application for providing heritage information and 3D visualization on a city scale. The LagunAR application was developed to provide historical information about the city of La Laguna in the XVI century, when it was the main city in the Canary Islands. The application provides a reconstructed 3D model of the city at that time that is shown on a mobile phone over-imposed on the actual city using geolocation. The geolocated position is used also for providing information of several points of interest in the city. The paper describes the design and implementation of the application and details the optimization techniques that have been used to manage the full information of the city using a mobile phone as a sensor and visualization tool. We explain the application usability study carried out using a heuristic test; in addition it is probed by users in a qualitative user test developed as preliminary research. Results show that it is possible to develop a real-time application that shows the user a city-scale 3D model and also manages the information of the points of interest. [ABSTRACT FROM AUTHOR]

The integration of Deep Learning (DL) models with the HoloLens2 Augmented Reality (AR) headset has enormous potential for real-time AR medical applications. Currently, most applications execute the models on an external server that communicates with the headset via Wi-Fi. This client-server architecture introduces undesirable delays and lacks reliability for real-time applications. However, due to HoloLens2's limited computation capabilities, running the DL model directly on the device and achieving real-time performances is not trivial. Therefore, this study has two primary objectives: (i) to systematically evaluate two popular frameworks to execute DL models on HoloLens2—Unity Barracuda and Windows Machine Learning (WinML)—using the inference time as the primary evaluation metric; (ii) to provide benchmark values for state-of-the-art DL models that can be integrated in different medical applications (e.g., Yolo and Unet models). In this study, we executed DL models with various complexities and analyzed inference times ranging from a few milliseconds to seconds. Our results show that Unity Barracuda is significantly faster than WinML (p-value < 0.005). With our findings, we sought to provide practical guidance and reference values for future studies aiming to develop single, portable AR systems for real-time medical assistance. [ABSTRACT FROM AUTHOR]

During the 2019 coronavirus disease pandemic, robotic-based systems for swab sampling were developed to reduce burdens on healthcare workers and their risk of infection. Teleoperated sampling systems are especially appreciated as they fundamentally prevent contact with suspected COVID-19 patients. However, the limited field of view of the installed cameras prevents the operator from recognizing the position and deformation of the swab inserted into the nasal cavity, which highly decreases the operating performance. To overcome this limitation, this study proposes a visual feedback system that monitors and reconstructs the shape of an NP swab using augmented reality (AR). The sampling device contained three load cells and measured the interaction force applied to the swab, while the shape information was captured using a motion-tracking program. These datasets were used to train a one-dimensional convolution neural network (1DCNN) model, which estimated the coordinates of three feature points of the swab in 2D X–Y plane. Based on these points, the virtual shape of the swab, reflecting the curvature of the actual one, was reconstructed and overlaid on the visual display. The accuracy of the 1DCNN model was evaluated on a 2D plane under ten different bending conditions. The results demonstrate that the x-values of the predicted points show errors of under 0.590 mm from P 0 , while those of P 1 and P 2 show a biased error of about −1.5 mm with constant standard deviations. For the y-values, the error of all feature points under positive bending is uniformly estimated with under 1 mm of difference, when the error under negative bending increases depending on the amount of deformation. Finally, experiments using a collaborative robot validate its ability to visualize the actual swab's position and deformation on the camera image of 2D and 3D phantoms. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) has been shown to improve productivity in industry, but its adverse effects (e.g., headaches, eye strain, nausea, and mental workload) on users warrant further investigation. The objective of this study is to investigate the effects of different instruction methods (i.e., HoloLens AR-based and paper-based instructions) and task complexity (low and high-demanding tasks) on cognitive workloads and performance. Twenty-eight healthy males with a mean age of 32.12 (SD 2.45) years were recruited in this study and were randomly divided into two groups. The first group performed the experiment using AR-based instruction, and the second group used paper-based instruction. Performance was measured using total task time (TTT). The cognitive workload was measured using the power of electroencephalograph (EEG) features and the NASA task load index (NASA TLX). The results showed that using AR instructions resulted in a reduction in maintenance times and an increase in mental workload compared to paper instructions, particularly for the more demanding tasks. With AR instruction, 0.45% and 14.94% less time was spent on low- and high-demand tasks, respectively, as compared to paper instructions. According to the EEG features, employing AR to guide employees during highly demanding maintenance tasks increased information processing, which could be linked with an increased germane cognitive load. Increased germane cognitive load means participants can better facilitate long-term knowledge and skill acquisition. These results suggested that AR is superior and recommended for highly demanding maintenance tasks since it speeds up maintenance times and increases the possibility that information is stored in long-term memory and encrypted for recalls. [ABSTRACT FROM AUTHOR]

Currently, strawberry harvesting relies heavily on human labour and subjective assessments of ripeness, resulting in inconsistent post-harvest quality. Therefore, the aim of this work is to automate this process and provide a more accurate and efficient way of assessing ripeness. We explored a unique combination of YOLOv7 object detection and augmented reality technology to detect and visualise the ripeness of strawberries. Our results showed that the proposed YOLOv7 object detection model, which employed transfer learning, fine-tuning and multi-scale training, accurately identified the level of ripeness of each strawberry with an mAP of 0.89 and an F1 score of 0.92. The tiny models have an average detection time of 18 ms per frame at a resolution of 1280 × 720 using a high-performance computer, thereby enabling real-time detection in the field. Our findings distinctly establish the superior performance of YOLOv7 when compared to other cutting-edge methodologies. We also suggest using Microsoft HoloLens 2 to overlay predicted ripeness labels onto each strawberry in the real world, providing a visual representation of the ripeness level. Despite some challenges, this work highlights the potential of augmented reality to assist farmers in harvesting support, which could have significant implications for current agricultural practices. [ABSTRACT FROM AUTHOR]

Normal-hearing people use sound as a cue to recognize various events that occur in their surrounding environment; however, this is not possible for deaf and hearing of hard (DHH) people, and in such a context they may not be able to freely detect their surrounding environment. Therefore, there is an opportunity to create a convenient device that can detect sounds occurring in daily life and present them visually instead of auditorily. Additionally, it is of great importance to appropriately evaluate how such a supporting device would change the lives of DHH people. The current study proposes an augmented-reality-based system for presenting household sounds to DHH people as visual information. We examined the effect of displaying both the icons indicating sounds classified by machine learning and a dynamic spectrogram indicating the real-time time–frequency characteristics of the environmental sounds. First, the issues that DHH people perceive as problems in their daily lives were investigated through a survey, suggesting that DHH people need to visualize their surrounding sound environment. Then, after the accuracy of the machine-learning-based classifier installed in the proposed system was validated, the subjective impression of how the proposed system increased the comfort of daily life was obtained through a field experiment in a real residence. The results confirmed that the comfort of daily life in household spaces can be improved by combining not only the classification results of machine learning but also the real-time display of spectrograms. [ABSTRACT FROM AUTHOR]

As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The digital transformation advancement enables multiple areas to provide modern services to their users. Culture is one of the areas that can benefit from these advances, more specifically museums, by presenting many benefits and the most emergent technologies to the visitors. This paper presents an indoor location system and content delivery solution, based on Bluetooth Low Energy Beacons, that enable visitors to walk freely inside the museum and receive augmented reality content based on the acquired position, which is done using the Received Signal Strength Indicator (RSSI). The solution presented in this paper was created for the Foz Côa Museum in Portugal and was tested in the real environment. A detailed study was carried out to analyze the RSSI under four different scenarios, and detection tests were carried out that allowed us to measure the accuracy of the room identification, which is needed for proper content delivery. Of the 89 positions tested in the four scenarios, 70% of the received signals were correctly received throughout the entire duration of the tests, 20% were received in an intermittent way, 4% were never detected and 6% of unwanted beacons were detected. The signal detection is fundamental for the correct room identification, which was performed with 96% accuracy. Thus, we verified that this technology is suitable for the proposed solution. [ABSTRACT FROM AUTHOR]

This paper proposes an algorithm for transmitting and reconstructing the estimated point cloud by temporally estimating a dynamic point cloud sequence. When a non-rigid 3D point cloud sequence (PCS) is input, the sequence is divided into groups of point cloud frames (PCFs), and a key PCF is selected. The 3D skeleton is predicted through 3D pose estimation, and the motion of the skeleton is estimated by analyzing the joints and bones of the 3D skeleton. For the deformation of the non-rigid human PC, the 3D PC model is transformed into a mesh model, and the key PCF is rigged using the 3D skeleton. After deforming the key PCF into the target PCF utilizing the motion vector of the estimated skeleton, the residual PC between the motion compensation PCF and the target PCF is generated. If there is a key PCF, the motion vector of the target PCF, and a residual PC, the target PCF can be reconstructed. Just as compression is performed using pixel correlation between frames in a 2D video, this paper compresses 3D PCFs by estimating the non-rigid 3D motion of a 3D object in a 3D PC. The proposed algorithm can be regarded as an extension of the 2D motion estimation of a rigid local region in a 2D plane to the 3D motion estimation of a non-rigid object (human) in 3D space. Experimental results show that the proposed method can successfully compress 3D PC sequences. If it is used together with a PC compression technique such as MPEG PCC (point cloud compression) in the future, a system with high compression efficiency may be configured. [ABSTRACT FROM AUTHOR]

Advancements in modern medicine have bolstered the usage of home-based rehabilitation services for patients, particularly those recovering from diseases or conditions that necessitate a structured rehabilitation process. Understanding the technological factors that can influence the efficacy of home-based rehabilitation is crucial for optimizing patient outcomes. As technologies continue to evolve rapidly, it is imperative to document the current state of the art and elucidate the key features of the hardware and software employed in these rehabilitation systems. This narrative review aims to provide a summary of the modern technological trends and advancements in home-based shoulder rehabilitation scenarios. It specifically focuses on wearable devices, robots, exoskeletons, machine learning, virtual and augmented reality, and serious games. Through an in-depth analysis of existing literature and research, this review presents the state of the art in home-based rehabilitation systems, highlighting their strengths and limitations. Furthermore, this review proposes hypotheses and potential directions for future upgrades and enhancements in these technologies. By exploring the integration of these technologies into home-based rehabilitation, this review aims to shed light on the current landscape and offer insights into the future possibilities for improving patient outcomes and optimizing the effectiveness of home-based rehabilitation programs. [ABSTRACT FROM AUTHOR]

In the context of web augmented reality (AR), 3D rendering that maintains visual quality and frame rate requirements remains a challenge. The lack of a dedicated and efficient 3D format often results in the degraded visual quality of the original data and compromises the user experience. This paper examines the integration of web-streamable view-dependent representations of large-sized and high-resolution 3D models in web AR applications. The developed cross-platform prototype exploits the batched multi-resolution structures of the Nexus.js library as a dedicated lightweight web AR format and tests it against common formats and compression techniques. Built with AR.js and Three.js open-source libraries, it allows the overlay of the multi-resolution models by interactively adjusting the position, rotation and scale parameters. The proposed method includes real-time view-dependent rendering, geometric instancing and 3D pose regression for two types of AR: natural feature tracking (NFT) and location-based positioning for large and textured 3D overlays. The prototype achieves up to a 46% speedup in rendering time compared to optimized glTF models, while a 34 M vertices 3D model is visible in less than 4 s without degraded visual quality in slow 3D networks. The evaluation under various scenes and devices offers insights into how a multi-resolution scheme can be adopted in web AR for high-quality visualization and real-time performance. [ABSTRACT FROM AUTHOR]

Despite the substantial progress achieved in the development and integration of augmented reality (AR) in surgical robotic and autonomous systems (RAS), the center of focus in most devices remains on improving end-effector dexterity and precision, as well as improved access to minimally invasive surgeries. This paper aims to provide a systematic review of different types of state-of-the-art surgical robotic platforms while identifying areas for technological improvement. We associate specific control features, such as haptic feedback, sensory stimuli, and human–robot collaboration, with AR technology to perform complex surgical interventions for increased user perception of the augmented world. Current researchers in the field have, for long, faced innumerable issues with low accuracy in tool placement around complex trajectories, pose estimation, and difficulty in depth perception during two-dimensional medical imaging. A number of robots described in this review, such as Novarad and SpineAssist, are analyzed in terms of their hardware features, computer vision systems (such as deep learning algorithms), and the clinical relevance of the literature. We attempt to outline the shortcomings in current optimization algorithms for surgical robots (such as YOLO and LTSM) whilst providing mitigating solutions to internal tool-to-organ collision detection and image reconstruction. The accuracy of results in robot end-effector collisions and reduced occlusion remain promising within the scope of our research, validating the propositions made for the surgical clearance of ever-expanding AR technology in the future. [ABSTRACT FROM AUTHOR]

In this work, two methods are proposed for solving the problem of one-dimensional barcode segmentation in images, with an emphasis on augmented reality (AR) applications. These methods take the partial discrete Radon transform as a building block. The first proposed method uses overlapping tiles for obtaining good angle precision while maintaining good spatial precision. The second one uses an encoder–decoder structure inspired by state-of-the-art convolutional neural networks for segmentation while maintaining a classical processing framework, thus not requiring training. It is shown that the second method's processing time is lower than the video acquisition time with a 1024 × 1024 input on a CPU, which had not been previously achieved. The accuracy it obtained on datasets widely used by the scientific community was almost on par with that obtained using the most-recent state-of-the-art methods using deep learning. Beyond the challenges of those datasets, the method proposed is particularly well suited to image sequences taken with short exposure and exhibiting motion blur and lens blur, which are expected in a real-world AR scenario. Two implementations of the proposed methods are made available to the scientific community: one for easy prototyping and one optimised for parallel implementation, which can be run on desktop and mobile phone CPUs. [ABSTRACT FROM AUTHOR]

The use of augmented reality (AR) technology is growing in the maintenance industry because it can improve efficiency and reduce costs by providing real-time guidance and instruction to workers during repairs and maintenance tasks. AR can also assist with equipment training and visualization, allowing users to explore the equipment's internal structure and size. The adoption of AR in maintenance is expected to increase as hardware options expand and development costs decrease. To implement AR for job aids in mobile applications, 3D spatial information and equipment details must be addressed, and calibrated using image-based or object-based tracking, which is essential for integrating 3D models with physical components. The present paper suggests a system using AR-assisted deep reinforcement learning (RL)-based model for NanoDrop Spectrophotometer training and maintenance purposes that can be used for rapid repair procedures in the Industry 4.0 (I4.0) setting. The system uses a camera to detect the target asset via feature matching, tracking techniques, and 3D modeling. Once the detection is completed, AR technologies generate clear and easily understandable instructions for the maintenance operator's device. According to the research findings, the model's target technique resulted in a mean reward of 1.000 and a standard deviation of 0.000 . This means that all the rewards that were obtained in the given task or environment were exactly the same. The fact that the reward standard deviation is 0.000 shows that there is no variability in the outcomes. [ABSTRACT FROM AUTHOR]

The successful implementation of Human–Robot Collaboration (HRC) has become a prominent feature of smart manufacturing environments. Key industrial requirements, such as flexibility, efficiency, collaboration, consistency, and sustainability, present pressing HRC needs in the manufacturing sector. This paper provides a systemic review and an in-depth discussion of the key technologies currently being employed in smart manufacturing with HRC systems. The work presented here focuses on the design of HRC systems, with particular attention given to the various levels of Human–Robot Interaction (HRI) observed in the industry. The paper also examines the key technologies being implemented in smart manufacturing, including Artificial Intelligence (AI), Collaborative Robots (Cobots), Augmented Reality (AR), and Digital Twin (DT), and discusses their applications in HRC systems. The benefits and practical instances of deploying these technologies are showcased, emphasizing the substantial prospects for growth and improvement in sectors such as automotive and food. However, the paper also addresses the limitations of HRC utilization and implementation and provides some insights into how the design of these systems should be approached in future work and research. Overall, this paper provides new insights into the current state of HRC in smart manufacturing and serves as a useful resource for those interested in the ongoing development of HRC systems in the industry. [ABSTRACT FROM AUTHOR]

This paper presents a novel unsupervised learning framework for estimating scene depth and camera pose from video sequences, fundamental to many high-level tasks such as 3D reconstruction, visual navigation, and augmented reality. Although existing unsupervised methods have achieved promising results, their performance suffers in challenging scenes such as those with dynamic objects and occluded regions. As a result, multiple mask technologies and geometric consistency constraints are adopted in this research to mitigate their negative impacts. Firstly, multiple mask technologies are used to identify numerous outliers in the scene, which are excluded from the loss computation. In addition, the identified outliers are employed as a supervised signal to train a mask estimation network. The estimated mask is then utilized to preprocess the input to the pose estimation network, mitigating the potential adverse effects of challenging scenes on pose estimation. Furthermore, we propose geometric consistency constraints to reduce the sensitivity of illumination changes, which act as additional supervised signals to train the network. Experimental results on the KITTI dataset demonstrate that our proposed strategies can effectively enhance the model's performance, outperforming other unsupervised methods. [ABSTRACT FROM AUTHOR]

Location-based Augmented Reality applications are increasingly used in many research and commercial fields. Some of the fields that these applications are used are recreational digital games, tourism, education, and marketing. This study aims to present a location-based augmented reality (AR) application for cultural heritage communication and education. The application was created to inform the public, especially K12 students, about a district of their city with cultural heritage value. Furthermore, Google Earth was utilized to create an interactive virtual tour for consolidating the knowledge acquired by the location-based AR application. A scheme for evaluating the AR application was also constructed using factors suitable for location-based applications: challenge, educational usefulness (knowledge), collaboration, and intention to reuse. A sample of 309 students evaluated the application. Descriptive statistical analysis showed that the application scored well in all factors, especially in challenge and knowledge (mean values 4.21 and 4.12). Furthermore, structural equation modeling (SEM) analysis led to a model construction that represents how the factors are causally related. Based on the findings, the perceived challenge significantly influenced the perceived educational usefulness (knowledge) (b = 0.459, sig = 0.000) and interaction levels (b = 0.645, sig = 0.000). Interaction amongst users also had a significant positive impact on users' perceived educational usefulness (b = 0.374, sig = 0.000), which in turn influenced users' intention to reuse the application (b = 0.624, sig = 0.000). [ABSTRACT FROM AUTHOR]

Image stitching is of great importance for multiple fields such as moving object detection and tracking, ground reconnaissance and augmented reality. To ameliorate the stitching effect and alleviate the mismatch rate, an effective image stitching algorithm based on color difference and an improved KAZE with a fast guided filter is proposed. Firstly, the fast guided filter is introduced to reduce the mismatch rate before feature matching. Secondly, the KAZE algorithm based on improved random sample consensus is used for feature matching. Then, the color difference and brightness difference of the overlapping area are calculated to make an overall adjustment to the original images so as to improve the nonuniformity of the splicing result. Finally, the warped images with color difference compensation are fused to obtain the stitched image. The proposed method is evaluated by both visual effect mapping and quantitative values. In addition, the proposed algorithm is compared with other current popular stitching algorithms. The results show that the proposed algorithm is superior to other algorithms in terms of the quantity of feature point pairs, the matching accuracy, the root mean square error and the mean absolute error. [ABSTRACT FROM AUTHOR]

When producing an engaging augmented reality (AR) user experience, it is crucial to create AR content that mimics real-life objects' behavior to the greatest extent possible. A critical aspect to achieve this is ensuring that the digital objects conform to line-of-sight rules and are either partially or completely occluded, just like real-world objects would be. The study explores the concept of utilizing a pre-existing 3D representation of the physical environment as an occlusion mask that governs the rendering of each pixel. Specifically, the research aligns a Level of Detail (LOD) 1 building model and a 3D mesh model with their real-world counterparts and evaluates the effectiveness of occlusion between the two models in an outdoor setting. Despite the mesh model containing more detailed information, the overall results do not show improvement. In an indoor scenario, the researchers leverage the scanning capability of HoloLens 2.0 to create a pre-scanned representation, which helps overcome the limited range and delay of the mesh reconstruction. [ABSTRACT FROM AUTHOR]

Maintenance of home appliances can be tedious. Maintenance work can be physically demanding and it is not always easy to know the cause of a malfunctioning appliance. Many users need to motivate themselves to perform maintenance work and consider it ideal for home appliances to be maintenance-free. On the other hand, pets and other living creatures can be taken care of with joy and without much pain, even if they are difficult to take care of. To alleviate the hassle associated with the maintenance of home appliances, we propose an augmented reality (AR) system to superimpose an agent over the home appliance of concern who changes their behavior according to the internal state of the appliance. Taking a refrigerator as an example, we verify whether such AR agent visualization motivates users to perform maintenance work and reduces the associated discomfort. We designed a cartoon-like agent and implemented a prototype system using a HoloLens 2, which can switch between several animations depending on the internal state of the refrigerator. Using the prototype system, a Wizard of Oz user study comparing three conditions was conducted. We compared the proposed method (Animacy condition), an additional behavior method (Intelligence condition), and a text-based method as a baseline for presenting the refrigerator state. In the Intelligence condition, the agent looked at the participants from time to time as if it was aware of them and exhibited help-seeking behavior only when it was considered that they could take a short break. The results show that both the Animacy and Intelligence conditions induced animacy perception and a sense of intimacy. It was also evident that the agent visualization made the participants feel more pleasant. On the other hand, the sense of discomfort was not reduced by the agent visualization and the Intelligence condition did not improve the perceived intelligence or the sense of coercion further compared to the Animacy condition. [ABSTRACT FROM AUTHOR]

The development of Industry 4.0 (I4.0) and the digitization and automation of manufacturing processes have created a demand for designing smart warehouses to support manufacturing processes. Warehousing is one of the fundamental processes in the supply chain, and is responsible for handling inventory. Efficient execution of warehouse operations often determines the effectiveness of realized goods flows. Therefore, digitization and its use in exchanging information between partners, especially real-time inventory levels, is critical. For this reason, the digital solutions of Industry 4.0 have quickly found application in internal logistics processes and enabled the design of smart warehouses, also known as Warehouse 4.0. The purpose of this article is to present the results of the conducted review of publications on the design and operation of warehouses using the concepts of Industry 4.0. A total of 249 documents from the last 5 years were accepted for analysis. Publications were searched for in the Web of Science database using the PRISMA method. The article presents in detail the research methodology and the results of the biometric analysis. Based on the results, a two-level classification framework was proposed, which includes 10 primary categories and 24 subcategories. Each of the distinguished categories was characterized based on the analyzed publications. It should be noted that in most of these studies, the authors' attention primarily focused on the implementation of (1) Industry 4.0 technological solutions, such as IoT, augmented reality, RFID, visual technology, and other emerging technologies; and (2) autonomous and automated vehicles in warehouse operations processes. Critical analysis of the literature also allowed us to identify the current research gaps, which will be the subject of further research by the authors. [ABSTRACT FROM AUTHOR]

The rapid development of information and communication technology has fostered a natural integration of technology and design. As a result, there is increasing interest in Augmented Reality (AR) business card systems that leverage digital media. This research aims to advance the design of an AR-based participatory business card information system in line with contemporary trends. Key aspects of this study include applying technology to acquire contextual information from paper business cards, transmitting it to a server, and delivering it to mobile devices; facilitating interactivity between users and content through a screen interface; providing multimedia business content (video, image, text, 3D elements) via image markers recognized by users on mobile devices, while also adapting the type and method of content delivery. The AR business card system designed in this research enhances traditional paper business cards by incorporating visual information and interactive elements and automatically generating buttons linked to phone numbers, location information, and homepages. This innovative approach enables users to interact and enriches their overall experience while adhering to strict quality control measures. [ABSTRACT FROM AUTHOR]

Accurate and robust camera pose estimation is essential for high-level applications such as augmented reality and autonomous driving. Despite the development of global feature-based camera pose regression methods and local feature-based matching guided pose estimation methods, challenging conditions, such as illumination changes and viewpoint changes, as well as inaccurate keypoint localization, continue to affect the performance of camera pose estimation. In this paper, we propose a novel relative camera pose regression framework that uses global features with rotation consistency and local features with rotation invariance. First, we apply a multi-level deformable network to detect and describe local features, which can learn appearances and gradient information sensitive to rotation variants. Second, we process the detection and description processes using the results from pixel correspondences of the input image pairs. Finally, we propose a novel loss that combines relative regression loss and absolute regression loss, incorporating global features with geometric constraints to optimize the pose estimation model. Our extensive experiments report satisfactory accuracy on the 7Scenes dataset with an average mean translation error of 0.18 m and a rotation error of 7.44° using image pairs as input. Ablation studies were also conducted to verify the effectiveness of the proposed method in the tasks of pose estimation and image matching using the 7Scenes and HPatches datasets. [ABSTRACT FROM AUTHOR]

Tremor is a movement disorder that significantly impacts an individual's physical stability and quality of life, and conventional medication or surgery often falls short in providing a cure. Rehabilitation training is, therefore, used as an auxiliary method to mitigate the exacerbation of individual tremors. Video-based rehabilitation training is a form of therapy that allows patients to exercise at home, reducing pressure on rehabilitation institutions' resources. However, it has limitations in directly guiding and monitoring patients' rehabilitation, leading to an ineffective training effect. This study proposes a low-cost rehabilitation training system that utilizes optical see-through augmented reality (AR) technology to enable tremor patients to conduct rehabilitation training at home. The system provides one-on-one demonstration, posture guidance, and training progress monitoring to achieve an optimal training effect. To assess the system's effectiveness, we conducted experiments comparing the movement magnitudes of individuals with tremors in the proposed AR environment and video environment, while also comparing them with standard demonstrators. Participants wore a tremor simulation device during uncontrollable limb tremors, with tremor frequency and amplitude calibrated to typical tremor standards. The results showed that participants' limb movement magnitudes in the AR environment were significantly higher than those in the video environment, approaching the movement magnitudes of the standard demonstrators. Hence, it can be inferred that individuals receiving tremor rehabilitation in the AR environment experience better movement quality than those in the video environment. Furthermore, participant experience surveys revealed that the AR environment not only provided a sense of comfort, relaxation, and enjoyment but also effectively guided them throughout the rehabilitation process. [ABSTRACT FROM AUTHOR]

Augmented reality and virtual reality technologies are witnessing an evolutionary change in the 5G and Beyond (5GB) network due to their promising ability to enable an immersive and interactive environment by coupling the virtual world with the real one. However, the requirement of low-latency connectivity, which is defined as the end-to-end delay between the action and the reaction, is very crucial to leverage these technologies for a high-quality immersive experience. This paper provides a comprehensive survey and detailed insight into various advantageous approaches from the hardware and software perspectives, as well as the integration of 5G technology, towards 5GB, in enabling a low-latency environment for AR and VR applications. The contribution of 5GB systems as an outcome of several cutting-edge technologies, such as massive multiple-input, multiple-output (mMIMO) and millimeter wave (mmWave), along with the utilization of artificial intelligence (AI) and machine learning (ML) techniques towards an ultra-low-latency communication system, is also discussed in this paper. The potential of using a visible-light communications (VLC)-guided beam through a learning algorithm for a futuristic, evolved immersive experience of augmented and virtual reality with the ultra-low-latency transmission of multi-sensory tracking information with an optimal scheduling policy is discussed in this paper. [ABSTRACT FROM AUTHOR]

The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are presented less frequently and usually focus on the description of the device rather than on its performance analysis. This is the case of projected augmented reality, which, compared to head-worn AR displays, offers the advantages of being simultaneously accessible by multiple users whilst preserving user awareness of the environment and feeling of immersion. This work provides a general evaluation of a custom-made head-mounted projector for the aid of precision manual tasks through an experimental protocol designed for investigating spatial and temporal registration and their combination. The results of the tests show that the accuracy ( 0.6 ± 0.1 mm of spatial registration error) and motion-to-photon latency ( 113 ± 12 ms) make the proposed solution suitable for guiding precision tasks. [ABSTRACT FROM AUTHOR]

The adoption of extended reality solutions is growing rapidly in the healthcare world. Augmented reality (AR) and virtual reality (VR) interfaces can bring advantages in various medical-health sectors; it is thus not surprising that the medical MR market is among the fastest-growing ones. The present study reports on a comparison between two of the most popular MR head-mounted displays, Magic Leap 1 and Microsoft HoloLens 2, for the visualization of 3D medical imaging data. We evaluate the functionalities and performance of both devices through a user-study in which surgeons and residents assessed the visualization of 3D computer-generated anatomical models. The digital content is obtained through a dedicated medical imaging suite (Verima imaging suite) developed by the Italian start-up company (Witapp s.r.l.). According to our performance analysis in terms of frame rate, there are no significant differences between the two devices. The surgical staff expressed a clear preference for Magic Leap 1, particularly for the better visualization quality and the ease of interaction with the 3D virtual content. Nonetheless, even though the results of the questionnaire were slightly more positive for Magic Leap 1, the spatial understanding of the 3D anatomical model in terms of depth relations and spatial arrangement was positively evaluated for both devices. [ABSTRACT FROM AUTHOR]

Researchers' interest in finding practical applications for virtual reality (VR) and augmented reality (AR) technologies has increased as new devices have become cheaper and more accessible, being used in entertainment, healthcare, and rehabilitation fields, among others. This study aims to provide an overview of the current state of scientific literature related to VR, AR, and physical activity (PA). A bibliometric analysis of studies published between 1994 and 2022 was conducted using The Web of Science (WoS), applying the traditional bibliometric laws and using the VOSviewer software for data and metadata processing. The results revealed an exponential increase in scientific production between 2009 and 2021 (R2 = 94%). The United States (USA) was the country/region with the most relevant co-authorship networks (72 papers); the most prolific author was Kerstin Witte, and the most prominent was Richard Kulpa. The most productive journal's core was composed of high-impact and open access journals. A great thematic diversity was found according to the most used keywords by the co-authors, highlighting concepts such as rehabilitation, cognition, training, and obesity. Then, the research related to this topic is in an exponential development phase, with great interest in the rehabilitation and sports sciences fields. [ABSTRACT FROM AUTHOR]

Older adults' independent life is compromised due to various problems, such as memory impairments and decision-making difficulties. This work initially proposes an integrated conceptual model for assisted living systems capable of providing helping means for older adults with mild memory impairments and their caregivers. The proposed model has four main components: (1) an indoor location and heading measurement unit in the local fog layer, (2) an augmented reality (AR) application to make interactions with the user, (3) an IoT-based fuzzy decision-making system to handle the direct and environmental interactions with the user, and (4) a user interface for caregivers to monitor the situation in real time and send reminders once required. Then, a preliminary proof-of-concept implementation is performed to evaluate the suggested mode's feasibility. Functional experiments are carried out based on various factual scenarios, which validate the effectiveness of the proposed approach. The accuracy and response time of the proposed proof-of-concept system are further examined. The results suggest that implementing such a system is feasible and has the potential to promote assisted living. The suggested system has the potential to promote scalable and customizable assisted living systems to reduce the challenges of independent living for older adults. [ABSTRACT FROM AUTHOR]

People with diabetes-related foot ulcers (DFUs) need to perform self-care consistently over many months to promote healing and to mitigate risks of hospitalisation and amputation. However, during that time, improvement in their DFU can be hard to detect. Hence, there is a need for an accessible method to self-monitor DFUs at home. We developed a new mobile phone app, "MyFootCare", to self-monitor DFU healing progression from photos of the foot. The aim of this study is to evaluate the engagement and perceived value of MyFootCare for people with a plantar DFU over 3 months' duration. Data are collected through app log data and semi-structured interviews (weeks 0, 3, and 12) and analysed through descriptive statistics and thematic analysis. Ten out of 12 participants perceive MyFootCare as valuable to monitor progress and to reflect on events that affected self-care, and seven participants see it as potentially valuable to enhance consultations. Three app engagement patterns emerge: continuous, temporary, and failed engagement. These patterns highlight enablers for self-monitoring (such as having MyFootCare installed on the participant's phone) and barriers (such as usability issues and lack of healing progress). We conclude that while many people with DFUs perceive app-based self-monitoring as valuable, actual engagement can be achieved for some but not for all people because of various facilitators and barriers. Further research should target improving usability, accuracy and sharing with healthcare professionals and test clinical outcomes when using the app. [ABSTRACT FROM AUTHOR]

Context: This review aimed to synthesize the literature on the acceptability, feasibility, and effectiveness of immersive virtual technologies to promote physical exercise in older people. Method: We performed a literature review, based on four databases (PubMed, CINAHL, Embase, and Scopus; last search: 30 January 2023). Eligible studies had to use immersive technology with participants aged 60 years and over. The results regarding acceptability, feasibility, and effectiveness of immersive technology-based interventions in older people were extracted. The standardized mean differences were then computed using a random model effect. Results: In total, 54 relevant studies (1853 participants) were identified through search strategies. Concerning the acceptability, most participants reported a pleasant experience and a desire to use the technology again. The average increase in the pre/post Simulator Sickness Questionnaire score was 0.43 in healthy subjects and 3.23 in subjects with neurological disorders, demonstrating this technology's feasibility. Regarding the effectiveness, our meta-analysis showed a positive effect of the use of virtual reality technology on balance (SMD = 1.05; 95% CI: 0.75–1.36; p < 0.001) and gait outcomes (SMD = 0.7; 95% CI: 0.14–0.80; p < 0.001). However, these results suffered from inconsistency and the number of trials dealing with these outcomes remains low, calling for further studies. Conclusions: Virtual reality seems to be well accepted by older people and its use with this population is feasible. However, more studies are needed to conclude its effectiveness in promoting exercise in older people. [ABSTRACT FROM AUTHOR]

New generation wearable devices allow for the development of interactive environments tailored for Virtual Reality (VR)– and Augmented Reality (AR)–based treatment of Autism Spectrum Disorders (ASD). Experts agree on their potential; however, there is lack of consensus on how to perform trials and the need arises for evaluation frameworks, methods, and techniques appropriate for the ASD population. In this paper, we report on a study conducted with high-functioning ASD people in the 21–23 age range, with the objectives of (1) evaluating the engagement of two headsets offering distinct immersive experiences, (2) reasoning on the interpretation of engagement factors in the case of ASD people, and (3) translating results into general guidelines for the development of VR/AR-based ASD treatment. To this aim, we (1) designed two engagement evaluation frameworks based on behavioral observation measures, (2) set up two packages of reference immersive scenarios, (3) defined the association between metrics and scenarios, and (4) administered the scenarios in distinct sessions for the investigated headsets. Results show that the immersive experiences are engaging and that the apparent lack of success of some evaluation factors can become potential advantages within the framework of VR/AR-based ASD treatment design. [ABSTRACT FROM AUTHOR]

Human action recognition systems use data collected from a wide range of sensors to accurately identify and interpret human actions. One of the most challenging issues for computer vision is the automatic and precise identification of human activities. A significant increase in feature learning-based representations for action recognition has emerged in recent years, due to the widespread use of deep learning-based features. This study presents an in-depth analysis of human activity recognition that investigates recent developments in computer vision. Augmented reality, human–computer interaction, cybersecurity, home monitoring, and surveillance cameras are all examples of computer vision applications that often go in conjunction with human action detection. We give a taxonomy-based, rigorous study of human activity recognition techniques, discussing the best ways to acquire human action features, derived using RGB and depth data, as well as the latest research on deep learning and hand-crafted techniques. We also explain a generic architecture to recognize human actions in the real world and its current prominent research topic. At long last, we are able to offer some study analysis concepts and proposals for academics. In-depth researchers of human action recognition will find this review an effective tool. [ABSTRACT FROM AUTHOR]

Medical ultrasound (US) is a commonly used modality for image-guided procedures. Recent research systems providing an in situ visualization of 2D US images via an augmented reality (AR) head-mounted display (HMD) were shown to be advantageous over conventional imaging through reduced task completion times and improved accuracy. In this work, we continue in the direction of recent developments by describing the first AR HMD application visualizing real-time volumetric (3D) US in situ for guiding vascular punctures. We evaluated the application on a technical level as well as in a mixed-methods user study with a qualitative prestudy and a quantitative main study, simulating a vascular puncture. Participants completed the puncture task significantly faster when using 3D US AR mode compared to 2D US AR, with a decrease of 28.4% in time. However, no significant differences were observed regarding the success rate of vascular puncture (2D US AR—50% vs. 3D US AR—72%). On the technical side, the system offers a low latency of 49.90 ± 12.92 ms and a satisfactory frame rate of 60 Hz. Our work shows the feasibility of a system that visualizes real-time 3D US data via an AR HMD, and our experiments show, furthermore, that this may offer additional benefits in US-guided tasks (i.e., reduced task completion time) over 2D US images viewed in AR by offering a vividly volumetric visualization. [ABSTRACT FROM AUTHOR]

This paper describes the deployment, integration, and demonstration of a Smartphone Video Guidance Sensor (SVGS) as a novel technology for autonomous 6-DOF proximity maneuvers and precision landing of a quadcopter drone. The proposed approach uses a vision-based photogrammetric position and attitude sensor (SVGS) to estimate the position of a landing target after video capture. A visual inertial odometry sensor (VIO) is used to provide position estimates of the UAV in a ground coordinate system during flight on a GPS-denied environment. The integration of both SVGS and VIO sensors enables the accurate updating of position setpoints during landing, providing improved performance compared with VIO-only landing, as shown in landing experiments. The proposed technique also shows significant operational advantages compared with state-of-the-art sensors for indoor landing, such as those based on augmented reality (AR) markers. [ABSTRACT FROM AUTHOR]

Human–Machine Interface (HMI) plays a key role in the interaction between people and machines, which allows people to easily and intuitively control the machine and immersively experience the virtual world of the meta-universe by virtual reality/augmented reality (VR/AR) technology. Currently, wearable skin-integrated tactile and force sensors are widely used in immersive human–machine interactions due to their ultra-thin, ultra-soft, conformal characteristics. In this paper, the recent progress of tactile and force sensors used in HMI are reviewed, including piezoresistive, capacitive, piezoelectric, triboelectric, and other sensors. Then, this paper discusses how to improve the performance of tactile and force sensors for HMI. Next, this paper summarizes the HMI for dexterous robotic manipulation and VR/AR applications. Finally, this paper summarizes and proposes the future development trend of HMI. [ABSTRACT FROM AUTHOR]

Breast cancer is the most prevalent cancer in the world and the fifth-leading cause of cancer-related death. Treatment is effective in the early stages. Thus, a need to screen considerable portions of the population is crucial. When the screening procedure uncovers a suspect lesion, a biopsy is performed to assess its potential for malignancy. This procedure is usually performed using real-time Ultrasound (US) imaging. This work proposes a visualization system for US breast biopsy. It consists of an application running on AR glasses that interact with a computer application. The AR glasses track the position of QR codes mounted on an US probe and a biopsy needle. US images are shown in the user's field of view with enhanced lesion visualization and needle trajectory. To validate the system, latency of the transmission of US images was evaluated. Usability assessment compared our proposed prototype with a traditional approach with different users. It showed that needle alignment was more precise, with 92.67 ± 2.32° in our prototype versus 89.99 ± 37.49° in a traditional system. The users also reached the lesion more accurately. Overall, the proposed solution presents promising results, and the use of AR glasses as a tracking and visualization device exhibited good performance. [ABSTRACT FROM AUTHOR]