Search

The traditional two-dimensional data visualization method is difficult to accurately express and display the three-dimensional information of the data, while the 3D interactive image technology can visually display the data in the threedimensional space. At the same time, it can also operate and edit the data through interactive methods to help people better understand and use the data. Intelligent algorithms are an important supplement to 3D interactive imaging technology, which can improve visualization and analysis accuracy through calculations, model building, and other methods, achieving more accurate data display and analysis. This article aimed to explore the application of intelligent algorithms and 3D interactive imaging technology in the field of spatial visualization. Intelligent algorithms can help analyze data features and relationships, extract data attributes, and optimize data display effects. 3D interactive imaging technology can present processed data in a virtual three-dimensional space, enhancing the interaction and visualization effect between users and data. The combination of the two can achieve better data analysis and presentation results. This article conducted simulation experiments on spatial visualization based on intelligent algorithms and 3D interactive imaging technology, and scored some experimental indicators using a scoring system. The experimental results of this article indicate that the intelligent algorithm had a time efficiency of 46.5 seconds and an accuracy of 95% in displaying spatial and distribution visualization. The usability score was 75. The interactivity score of 3D interactive imaging technology in displaying spatial and distribution visualization was 96. The visual effect score was 96, and the usability score was 65. The combination of the two in terms of displaying spatial and distribution visualization had a time efficiency of 50.8 seconds, interactivity of 86 minutes, and accuracy of 98 %. It shows that different data visualization technologies have their own characteristics in display space and distribution visualization, and suitable technologies can be selected according to actual needs and application scenarios. [ABSTRACT FROM AUTHOR]

Maintenance, storage and warehousing are complex processes required in many industries such as automotive, aerospace, manufacturing and logistic companies. These processes, often, involve moving objects in crowded environments using robots or human operators. Particularly, replacement and assembly of machine parts in crowded environments when performed by a human being require high technical skills. These tasks may be performed using robots to reduce costs due to human errors and execution time. However, robots under open world assumptions could neither operate in all environments nor perform tasks not modeled by the designer. In this paper, we introduce a mixed reality system to assist human operators in moving objects in crowded environments for maintenance tasks such as: parts assembly or replacement, and storage of objects. The introduced system consists of a mobile application exploited through a hands-free VR box. The proposed Mixed Reality for Industrial Maintenance (MRIM) system enhances the perception of a human operator by overlaying 3D real world visual information and virtual objects, such as: orientation guidelines including rotating angles, moving direction and displacement of carried objects. These guidelines allow for gaining execution time, and reducing human errors that might cause industrial parts damage. The proposed work brings two main contributions. First, it makes use of a new algorithm based on recasting, named R star (R*) that allows for optimizing pathfinding in 3D space. This later outperforms the two commonly used baseline 3D pathfinding algorithms of at least 87.5% in terms of execution time. Second, MRIM provides an easy-to-use interface that exploits information provided by the R* algorithm. The experiments, conducted in real condition for the task of part replacement in a crowded environment, show that MRIM reduces considerably execution time and human errors. [ABSTRACT FROM AUTHOR]

Six degrees of freedom (6DoF) input interfaces are essential formanipulating virtual objects through translation or rotation in three-dimensional (3D) space. A traditional outside-in tracking controller requires the installation of expensive hardware in advance. While inside-out tracking controllers have been proposed, they often suffer from limitations such as interaction limited to the tracking range of the sensor (e.g., a sensor on the head-mounted display (HMD)) or the need for pose valuemodification to function as an input interface (e.g., a sensor on the controller). This study investigates 6DoF pose estimation methods without restricting the tracking range, using a smartphone as a controller in augmented reality (AR) environments. Our approach involves proposing methods for estimating the initial pose of the controller and correcting the pose using an inside-out tracking approach. In addition, seven pose estimation algorithms were presented as candidates depending on the tracking range of the device sensor, the tracking method (e.g., marker recognition, visual-inertial odometry (VIO)), and whether modification of the initial pose is necessary. Through two experiments (discrete and continuous data), the performance of the algorithms was evaluated. The results demonstrate enhanced final pose accuracy achieved by correcting the initial pose. Furthermore, the importance of selecting the tracking algorithm based on the tracking range of the devices and the actual input value of the 3D interaction was emphasized. [ABSTRACT FROM AUTHOR]

The ability to track handheld devices in three-dimensional (3D) space is basic and critical in virtual reality (VR) and other systems. Although outside-in tracking solutions for six-degrees-of-freedom (6DoF) input are stable, such passive tracking systems are often expensive and require additional space and time to set up the base stations. We focus on an inside-out 6DoF input prototype based on Visual-Inertial Simultaneous Localization and Mapping (VI-SLAM), which is easy to deploy. For a long-term use of the inside-out tracker, auto-calibration is required to deal with the pose jump caused by relocalization or loop closing in VI-SLAM. We study the auto-calibration for the interaction in virtual reality to check if the pose jump interrupts the user experience. We further introduce a pose smoothing method to make the interaction more natural and coherent. Our user experiments have validated that the proposed inside-out input device with smooth auto-calibration had the comparable performance with existing commercial controllers for 6DoF VR interaction. [ABSTRACT FROM AUTHOR]

One of the key interactions in 3D environments is target acquisition, which can be challenging when targets are small or in cluttered scenes. Here, incorrect elements may be selected, leading to frustration and wasted time. The accuracy is further hindered by the physical act of selection itself, typically involving pressing a button. This action reduces stability, increasing the likelihood of erroneous target acquisition. We focused on molecular visualization and on the challenge of selecting atoms, rendered as small spheres. We present two techniques that improve upon previous progressive selection techniques. They facilitate the acquisition of neighbors after an initial selection, providing a more comfortable experience compared to using classical ray-based selection, particularly with occluded elements. We conducted a pilot study followed by two formal user studies. The results indicated that our approaches were highly appreciated by the participants. These techniques could be suitable for other crowded environments as well.

Virtual trackball techniques are widely used when 3D interaction is performed through interfaces with a reduced number of degrees of freedom such as mice and touchscreens. For decades, most implementations fix a vertical axis of rotation, which is a suitable choice when the vertical axis should indeed be fixed, according to some mental model of the user. We conducted an experiment involving the use of a mouse and a touch device to study usability in terms of performance, perceived usability and mental workload when selecting different fixed axes in accordance with the user's mental model. The results we obtained indicate that the consistency between the axis fixed by the technique and the object's intrinsic axis has a positive effect on usability. We believe that implementations that allow to select different fixed axis for each specific object should be considered when designing future reduced-DoF interaction interfaces. [ABSTRACT FROM AUTHOR]

A three-dimensional (3D) cursor for autostereoscopic 3D displays was proposed. The cursor is based on voxel patterns. In the experiments, the cursor position is interactively controlled in 3D by a computer mouse with a scroll wheel. A computer program operates in the image plane of the 3D display directly without complicated geometric calculations. The results were verified in a 3D display with a parallax barrier and can be applied to other types of autostereoscopic displays.

This paper presents an interactive three‐dimensional (3D) touch and gesture user interface that utilizes a light‐field display and a 3D sensor. In contrast to conventional gesture interfaces, in which usually the image space and gesture space are separated, a user can directly interact with the 3D image in this interface system. To achieve this trait, the 3D registration between the sensor and display coordinate systems is necessary. In this study, we propose a method to automatically match the position of the reconstructed 3D image with the position of the hand of the user based on scattered light detection. The proposed calibration process is automatic and enables self‐calibration during practical operation without laborious measurement work. The method allows us to use the gesture detection capabilities of the leap motion controller to develop an interface where content and gesture positions match. We show some examples of 3D touch and gesture interactions in which the gesture position and the generated content match on a light field display. [ABSTRACT FROM AUTHOR]

Over decades, many researchers developed complex in-lab systems with the overall goal to track multiple body parts of the user for a richer and more powerful 2D/3D interaction with a distant display. In this work, we introduce a novel smartphone-based tracking approach that eliminates the need for complex tracking systems. Relying on simultaneous usage of the front and rear smartphone cameras, our solution enables rich spatial interactions with distant displays by combining touch input with hand-gesture input, body and head motion, as well as eye-gaze input. In this paper, we firstly present a taxonomy for classifying distant display interactions, providing an overview of enabling technologies, input modalities, and interaction techniques, spanning from 2D to 3D interactions. Further, we provide more details about our implementation—using off-the-shelf smartphones. Finally, we validate our system in a user study by a variety of 2D and 3D multimodal interaction techniques, including input refinement. [ABSTRACT FROM AUTHOR]

A three-dimensional (3D) cursor for autostereoscopic 3D displays was proposed. The cursor is based on voxel patterns. In the experiments, the cursor position is interactively controlled in 3D by a computer mouse with a scroll wheel. A computer program operates in the image plane of the 3D display directly without complicated geometric calculations. The results were verified in a 3D display with a parallax barrier and can be applied to other types of autostereoscopic displays. [ABSTRACT FROM AUTHOR]

We propose IM3D+, a novel approach to reconstructing 3D motion data from a flexible magnetic flux sensor array using deep learning and a structure-aware temporal bilateral filter. Computing the 3D configuration of markers (inductor-capacitor (LC) coils) from flux sensor data is difficult because the existing numerical approaches suffer from system noise, dead angles, the need for initialization, and limitations in the sensor array’s layout. We solve these issues with deep neural networks to learn the regression from the simulation flux values to the LC coils’ 3D configuration, which can be applied to the actual LC coils at any location and orientation within the capture volume. To cope with the influence of system noise and the dead-angle limitation caused by the characteristics of the hardware and sensing principle, we propose a structure-aware temporal bilateral filter for reconstructing motion sequences. Our method can track various movements, including fingers that manipulate objects, beetles that move inside a vivarium with leaves and soil, and the flow of opaque fluid. Since no power supply is needed for the lightweight wireless markers, our method can robustly track movements for a very long time, making it suitable for various types of observations whose tracking is difficult with existing motion-tracking systems. Furthermore, the flexibility of the flux sensor layout allows users to reconfigure it based on their own applications, thus making our approach suitable for a variety of virtual reality applications. [ABSTRACT FROM AUTHOR]

Assembly planning is still a manual undertaking today. In past work, however, a novel approach was presented that allows precise assembly sequence generation on the basis of a variety of optimization criteria. This approach was integrated directly into a CAD system, which in turn results in a dependency on the CAD system. In the context of this work, it was therefore examined how the same result can be realized with a neutral approach. Therefore, it will be investigated what kind of information can be extracted directly and how the missing information can be added via a user interface. Therefore, a reverse engineering algorithm and an additional user interface were developed using a game engine. After the 3D model was analyzed, the 3D model will be imported into the user interface and the interactive disassembly of the model will begin. Thus, the product to disassemble is selected via mouse clicks and moved to its final position. Once the disassembly is finished, the generated information is saved into a neutral exchange format. Based on the exchange format, the system is able to generate automatically assembly animations and assembly videos. [ABSTRACT FROM AUTHOR]

As a hyper-natural interaction technique in 3D user interfaces, non-isomorphic rotation has been considered an effective approach for rotation tasks, where a static or dynamic control-display gain can be applied to amplify or attenuate a rotation. However, it is not clear whether non-isomorphic rotation can benefit 6-degree-of-freedom (6-DOF) manipulation tasks in AR and VR. In this article, we extended the usability studies of non-isomorphic rotation from rotation-only tasks to 6-DOF manipulation tasks and analyzed the collected data using a 2-component model. Using a mixed reality (MR) simulation approach, we also investigated whether environment (AR or VR) had an impact on 3D manipulation tasks. The results reveal that although both static and dynamic non-isomorphic rotation techniques could save time and effort in ballistic phases, only dynamic non-isomorphic rotation was significantly faster than isomorphic rotation. Interestingly, while environment had no significant impact on overall user performance, we found evidence that it could affect fine-tuning in correction phases. We also found that most participants preferred AR over VR, indicating that environmental visual realism could be helpful to improve user experience. [ABSTRACT FROM AUTHOR]

In the last decade, there has been an extraordinary acceleration in immersive technologies, including virtual reality (VR) and augmented reality (AR). The VR interfaces have widely explored various 3D interaction‐based approaches. 3D interaction is one of the main features of any VR system. However, the problems while selecting and manipulating distant and occluded 3D virtual objects in VR are still unresolved and suffer from a lack of precision and accuracy. To interact with VR/AR systems, various devices such as VR headsets or gloves are used. These devices are not very reliable yet; in fact they are inconvenient and invasive to a person's comfort. Currently, the natural user interfaces (NUIs) are increasingly introduced in human computer interaction (HCI) systems as gestures recognition, which prove to be very useful in the improvement of the user engagement and presence sensing. More particularly, they provide more user‐friendly and nonintrusive 3D interaction methods and techniques. In this article, the Zoom‐fwd, which is an efficient 3D interaction technique is presented. The proposed technique uses gesture recognition for different 3D interaction tasks like selection and manipulation. This new approach allows an efficient interaction with distant and occluded objects, while providing a precise selection, even when the environment is crowded. The Zoom‐fwd technique is a software solution for a number of hardware and software problems. A user study is conducted to determine whether the proposed technique is more suitable when performing interaction tasks. The results show that the Zoom‐fwd technique provides effective interaction with distant and occluded objects, by improving the user task completion performance. In addition, this indicates that selection precision has been enhanced significantly with the Zoom‐fwd technique. [ABSTRACT FROM AUTHOR]

Virtual reality drawing applications let users draw 3D shapes using brushes that form ribbon shaped, or ruled-surface, strokes. Each ribbon is uniquely defined by its user-specified ruling length, path, and the ruling directions at each point along this path. Existing brushes use the trajectory of a handheld controller in 3D space as the ribbon path, and compute the ruling directions using a fixed mapping from a specific controller coordinate-frame axis. This fixed mapping forces users to rotate the controller and thus their wrists to change ribbon normal or ruling directions, and requires substantial physical effort to draw even medium complexity ribbons. Since human ability to rotate their wrists continuously is heavily restricted, the space of ribbon geometries users can comfortably draw using these brushes is limited. These brushes can be unpredictable, producing ribbons with unexpectedly varying width or flipped and wobbly normals in response to seemingly natural hand gestures. Our AdaptiBrush ribbon brush system dramatically extends the space of ribbon geometries users can comfortably draw while enabling them to accurately predict the ribbon shape that a given hand motion produces. We achieve this by introducing a novel adaptive ruling direction computation method, enabling users to easily change ribbon ruling and normal orientation using predominantly translational controller, and thus wrist, motion. We facilitate ease-of-use by computing predictable ruling directions that smoothly change in both world and controller coordinate systems, and facilitate ease-of-learning by prioritizing ruling directions which are well-aligned with one of the controller coordinate system axes. Our comparative user studies confirm that our more general and predictable ruling computation leads to significant improvements in brush usability and effectiveness compared to all prior brushes; in a head to head comparison users preferred AdaptiBrush over the next-best brush by a margin of 2 to 1. [ABSTRACT FROM AUTHOR]

Over decades, many researchers developed complex in-lab systems with the overall goal to track multiple body parts of the user for a richer and more powerful 2D/3D interaction with a distant display. In this work, we introduce a novel smartphone-based tracking approach that eliminates the need for complex tracking systems. Relying on simultaneous usage of the front and rear smartphone cameras, our solution enables rich spatial interactions with distant displays by combining touch input with hand-gesture input, body and head motion, as well as eye-gaze input. In this paper, we firstly present a taxonomy for classifying distant display interactions, providing an overview of enabling technologies, input modalities, and interaction techniques, spanning from 2D to 3D interactions. Further, we provide more details about our implementation—using off-the-shelf smartphones. Finally, we validate our system in a user study by a variety of 2D and 3D multimodal interaction techniques, including input refinement.

Hand interaction techniques in virtual reality often exploit visual dominance over proprioception to remap physical hand movements onto different virtual movements. However, when the offset between virtual and physical hands increases, the remapped virtual hand movements are hardly self-attributed, and the users become aware of the remapping. Interestingly, the sense of self-attribution of a body is called the sense of body ownership (SoBO) in the field of psychology, and the realistic the avatar, the stronger is the SoBO. Hence, we hypothesized that realistic avatars (i.e., human hands) can foster self-attribution of the remapped movements better than abstract avatars (i.e., spherical pointers), thus making the remapping less noticeable. In this article, we present an experiment in which participants repeatedly executed reaching movements with their right hand while different amounts of horizontal shifts were applied. We measured the remapping detection thresholds for each combination of shift directions (left or right) and avatar appearances (realistic or abstract). The results show that realistic avatars increased the detection threshold (i.e., lowered sensitivity) by 31.3 percent than the abstract avatars when the leftward shift was applied (i.e., when the hand moved in the direction away from the body-midline). In addition, the proprioceptive drift (i.e., the displacement of self-localization toward an avatar) was larger with realistic avatars for leftward shifts, indicating that visual information was given greater preference during visuo-proprioceptive integration in realistic avatars. Our findings quantifiably show that realistic avatars can make remapping less noticeable for larger mismatches between virtual and physical movements and can potentially improve a wide variety of hand-remapping techniques without changing the mapping itself. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) is a widely researched route for navigation support in laparoscopic surgery. Accurate registration is a crucial component for such AR systems. We introduce two methods for interactive registration that aim to be minimally invasive to the workflow and to mimic natural manipulation of 3D objects. The methods utilise spatially tracked laparoscopic tools to manipulate the virtual 3D content. We comparatively evaluated the methods against a reference, landmark-based registration method in a user study with 12 participants. We tested the methods for registration accuracy, time, and subjective usability perception. Our methods did not outperform the reference method on these parameters but showed promising results. The results indicate that our methods present no finalised solutions but that one of them is a promising approach for which we identified concrete improvement measures to be implemented in future research. [ABSTRACT FROM AUTHOR]

The input method based on free-hand gestures has gradually become a hot research direction in the field of human-computer interaction. Hand gestures like sign languages, however, demand quite a lot of knowledge and practice for interaction, and air writing methods require their users to hold the arm and hand in mid-air for a period of time. These methods limit the user experience and get severer when a large number of gestures are required. To address the problem, this paper presents a novel human-3DTV interaction system based on a set of simple free-hand gestures for direct-touch interaction with a virtual interface to facilitate human-3DTV interaction. Specifically, our system projects a virtual interface in front of the user who wears the 3D shutter glass, and the user just stretches the arm and touches the virtual interface like performing on a smart phone with a touch screen, using gestures such as Click, Slide, Hold, Drag and Zoom In/Out. Our system is able to recognize the user's gesture fast and accurately, as the system only needs to search for a small region neighboring the virtual interface for a small set of gesture types. Because we adopt the key gestures using on smart phones, our free-hand gestures can be easily used by anyone with only a brief training. The users feel more comfortable than traditional gesture input methods and can effectively interact with 3DTV using our system. We report a comprehensive user study on accuracy and speed to validate the advantages of the proposed human-3DTV interaction system.

AbstractWe present a novel selection technique for VR called LenSelect. The main idea is to decrease the Index of Difficulty (ID) according to Fitts’ Law by dynamically increasing the size of the potentially selectable objects. This facilitates the selection process especially in cases of small, distant or partly occluded objects, but also for moving targets. In order to evaluate our method, we have defined a set of test scenarios that covers a broad range of use cases, in contrast to often used simpler scenes. Our test scenarios include practically relevant scenarios with realistic objects but also synthetic scenes, all of which are available for download. We have evaluated our method in a user study and compared the results to two state-of-the-art selection techniques and the standard ray-based selection. Our results show that LenSelect performs similar to the fastest method, which is ray-based selection, while significantly reducing the error rate by 44%.

• Development and user assessment of an Augmented Situated Visualization of EMC data. • Significant effect of the interaction methods on the time taken for data extraction. • Significant effect of the interaction methods used on walking steps. • Significant effect of the interaction and visualization methods on task correctness. • Results suggest that specific techniques may be used depending on the task requisites. In electrical engineering, hardware experts often need to analyze electromagnetic radiation data to detect any external interference or anomaly. The field that studies this sort of assessment is called electromagnetic compatibility (EMC). As a way to support EMC analysis, we propose the use of Augmented Situated Visualization (ASV) to supply professionals with visual and interactive information that helps them to comprehend that data, however situating it where it is most relevant in its spatial context. Users are able to interact with the visualization by changing the attributes being displayed, comparing the overlaps of multiple fields, and extracting data, as a way to refine their search. The solutions being proposed in this work were tested against each other in comparable 2D and 3D interactive visualizations of the same data in a series of data-extraction assessments with users, as a means to validate the approaches. Results exposed a correctness-time trade-off between the interaction methods. The hand-based techniques (Hand Slider and Touch Lens) were the least error-prone, being near to half as error-inducing as the gaze-based method. Touch Lens also performed as the least time-consuming method, taking in average less than half of the average time required by the others. For the visualization methods tested, the 2D ray casts presented a higher usability score and lesser workload index than the 3D topology view , however exposing over two times the error ratio. Ultimately, this work exposes how AR can help users to have better performances in a decision-making context, particularly in EMC related tasks, while also furthering the research in the ASV field. [ABSTRACT FROM AUTHOR]

The selection and manipulation of objects in Virtual Reality face application developers with a substantial challenge as they need to ensure a seamless interaction in three-dimensional space. Assessing the advantages and disadvantages of selection and manipulation techniques in specific scenarios and regarding usability and user experience is a mandatory task to find suitable forms of interaction. In this article, we take a look at the most common issues arising in the interaction with objects in VR. We present a taxonomy allowing the classification of techniques regarding multiple dimensions. The issues are then associated with these dimensions. Furthermore, we analyze the results of a study comparing multiple selection techniques and present a tool allowing developers of VR applications to search for appropriate selection and manipulation techniques and to get scenario dependent suggestions based on the data of the executed study. [ABSTRACT FROM AUTHOR]