Search

Turbulent vortices in smoke flows are crucial for a visually interesting appearance. Unfortunately, it is challenging to efficiently simulate these appealing effects in the framework of vortex filament methods. The vortex filaments in grids scheme allows to efficiently generate turbulent smoke with macroscopic vortical structures, but suffers from the projection-related dissipation, and thus the small-scale vortical structures under grid resolution are hard to capture. In addition, this scheme cannot be applied in wall-bounded turbulent smoke simulation, which requires efficiently handling smoke-obstacle interaction and creating vorticity at the obstacle boundary. To tackle above issues, we propose an effective filament-mesh particle-particle (FMPP) method for fast wall-bounded turbulent smoke simulation with ample details. The Filament-Mesh component approximates the smooth long-range interactions by splatting vortex filaments on grid, solving the Poisson problem with a fast solver, and then interpolating back to smoke particles. The Particle-Particle component introduces smoothed particle hydrodynamics (SPH) turbulence model for particles in the same grid, where interactions between particles cannot be properly captured under grid resolution. Then, we sample the surface of obstacles with boundary particles, allowing the interaction between smoke and obstacle being treated as pressure forces in SPH. Besides, the vortex formation region is defined at the back of obstacles, providing smoke particles flowing by the separation particles with a vorticity force to simulate the subsequent vortex shedding phenomenon. The proposed approach can synthesize the lost small-scale vortical structures and also achieve the smoke-obstacle interaction with vortex shedding at obstacle boundaries in a lightweight manner. The experimental results demonstrate that our FMPP method can achieve more appealing visual effects than vortex filaments in grids scheme by efficiently simulating more vivid thin turbulent features.

Numerical fractures severely impact the visual effects of SPH based free-surface liquid simulation, leading to surface tension artifact, particles clumping near the free-surface and thin features rupture in the free-surface. To tackle this issue, we first introduce a fast and accurate geometrical method to detect the free-surface particles and compute their normals for precise surface tension computation, avoiding surface-tension-like artifacts. Furthermore, we define a level-set function for each particle configuration and correct the miscalculated density induced by numerical fractures at free surface and solid boundary. In addition, we integrate our method into the PCISPH solver, which makes the SPH particles distributed more homogeneously, thereby improving the stability of the numerical simulation. We evaluate our method on a variety of benchmark examples, and experimental results show the proposed method can robustly, efficiently and realistically simulate the SPH-based free surface liquids.

Haptic feedback is crucial to immersive experience in virtual and augmented reality applications. The existing promising maglev haptic devices have advantages of no mechanical friction and low inertia. However, their performance is limited by the navigation approach, which mainly results from the challenge that it is difficult to obtain high precision, high frequency, and good stability with lightweight design at the same time. In this study, we reformulate visual-inertial navigation as a regression problem, and adopt deep learning to perform fusion navigation for maglev haptic interaction. A cascade-LSTM-based q-increment learning method is first proposed to progressively learn the increments of target variables. Two cascade LSTM networks are then constructed to estimate the increments of position and orientation, which are pipelined to accomplish visual-inertial fusion navigation. Additionally, we set up a maglev haptic platform as the system testbed. Experimental results show that our cascade-LSTMbased visual-inertial fusion navigation approach can reach 200 Hz while maintaining high-precision navigation (the mean absolute error of the position and orientation is less than 1 mm and 0.02°, respectively) for a maglev haptic interactive deformation application.

Realistic liquid animations with thin sheets or streams are crucial for creating fluid effects in digital media. However, it is challenging to simulate these appealing thin sheets or streams in the framework of smoothed particle hydrodynamics (SPH). The underlying reason for this challenge mainly lies in the inherent numerical instability of SPH due to inconsistent kernel interpolation, which is caused by the incomplete kernel support on the free surface and the particles’ disorder dispersion within the simulation domain. To address this challenge, we propose a novel and effective approach to ensure the consistency of kernel interpolation at both internal flow and the free surface during the simulation such that these thin features can always be well maintained. First, we introduce a transport-velocity formulation to alleviate the disorder dispersion in the liquid domain. However, this formulation can only work in the internal flow, and it fails at the free surface because it cannot accurately estimate the density of particles there. To this end, we propose adaptively correcting the underestimated density caused by the incomplete kernel support of free-surface particles, which are identified by a geometry-aware anisotropic kernel, to counteract the inconsistent interpolation on the free surface. Then, we propose a novel scheme to further filter the background pressure to enhance the interactions between the internal flow and the free surface, as well as liquid and solid, such that the thin features generated from such interactions can be realistically simulated. The proposed approach can also achieve anticlumping and regularization effects in the entire simulation domain and, hence, further enhance the thin features in liquids. We evaluate our method on a variety of benchmark examples, and the results demonstrate that our method can achieve more appealing visual effects than state-of-the-art methods by realistically simulating more vivid thin features.

Microelectrode recordings (MERs) are a significant clinical indicator for sweet spots identification of implanted electrodes during deep brain stimulation of the subthalamic nucleus (STN) surgery. As 1D MERs signals have the unboundedness, large-range, large-amount and time-dependent characteristics, the purpose of this study is to propose an automatic and precise identification method of sweet spots from MERs, reducing the time-consuming and labor-intensive human annotations. We propose an automatic identification method of sweet spots from MERs for electrodes implantation in STN-DBS. To better imitate the surgeons’ observation and obtain more intuitive contextual information, we first employ the 2D Gramian angular summation field (GASF) images generated from MERs data to perform the sweet spots determination for electrodes implantation. Then, we introduce the convolutional block attention module into convolutional neural network (CNN) to identify the 2D GASF images of sweet spots for electrodes implantation. Experimental results illustrate that the identification result of our method is consistent with the result of doctor’s decision, while our method can achieve the accuracy and precision of 96.72% and 98.97%, respectively, which outperforms state-of-the-art for intraoperative sweet spots determination. The proposed method is the first time to automatically and accurately identify sweet spots from MERs for electrodes implantation by the combination an advanced time series-to-image encoding way with CBAM-enhanced networks model. Our method can assist neurosurgeons in automatically detecting the most likely locations of sweet spots for electrodes implantation, which can provide an important indicator for target selection while it reduces the localization error of the target during STN-DBS surgery.

This paper presents a novel mixed reality (MR) guidance method for liver tumors radiofrequency ablation (RFA). Compared with traditional computed tomography (CT)-guided method, our system can provide a more natural and intuitive surgical mode for surgeons. In essence, our system is a holographic navigation platform, which projects a MR overlay onto the patient via HoloLens during RFA. We first reconstruct the patient-specific anatomy structure from the CT images of abdominal phantom. Then, a tailored precise registration method is employed to map the virtual-real spatial information. In addition, considering that tumor shifting during biopsy severely impacts the accuracy of RFA, our guidance system involves a motion compensation computation through data-driven physically-based modeling in holographic environment. In experiments, we conduct a user study on the comparison trial between MR-guided and CT-guided biopsy. User feedback demonstrates that our MR guidance method for needle placement procedure has the potential to simplify the operation, reduce the operation difficulty, shorten the operation time, and raise the operation precision.

Interactive progressive cutting of deformable body is vital for surgical training/planning systems with resection operations. Unfortunately, it is challenging to robustly simulate this phenomenon with faithful reflection of user's motion. The underlying reason is that sophisticated boundary conditions should be considered for the consistency between scalpel motion trajectory and incision during progressive cutting. Meanwhile, both geometrical and physical primitives along cutting trajectory need to be updated for each step induced by topological modifications, which is time-consuming for interactive environment. To tackle the above-mentioned issues, we present a phyxel-associated surface mesh approach for interactive progressive cutting of deformable bodies. Our method is based on a hybrid geometry, including a triangular surface mesh and an internal physical point cloud (phyxels), and we employ the moving least squares shape function to achieve the fast bijective mapping. This structure can well decouple the mechanical modeling from the geometry processing, thus allowing efficient handling of topological modifications, mechanical response, and rendering issues. Furthermore, we propose tailored continuous collision detection and constrained bilateral triangulation methods for fast geometrical modification. Besides, a dynamic adaptation scheme is proposed to avoid material slivers along splitting trajectories and guarantee numerical stability of simulation. Experimental results demonstrate that our method can provide effective progressive cutting simulation with realistic mechanical response, and it has a great potential to be an efficient and versatile computational framework for interactive surgical cutting simulation with high fidelity.

At present, there are many researches in EEG sleep staging, but most of them have time-consuming or space complexity problems and are not suitable for the development of wearable devices. In this paper, a multi-level sleep staging method based on amplitude-time mapping is proposed to improve the accuracy and practicality. Firstly, by mapping the signal to amplitude axis and time axis respectively, we can obtain the sleep feature information related to EEG variation characteristic. Then, the classification features of different sleep stages are extracted, and the complex signal features representing non-linearity can be obtained by combining the extracted features. Finally, according to the differences and similarities of EEG in different sleep stages, a classification model of multi-level weighted support vector machine (M-WSVM) is established. The experiment results from two EEG sleep datasets show that the proposed method has better sleep staging performance and can effectively promote the development and application of wearable EEG sleep staging system.

As a cutting-edge research topic in computer vision and graphics for decades, human skeleton extraction from single-depth camera remains challenging due to possibly occurring occlusions of different body parts, huge appearance variations, and sensor noise. In this paper, we propose to incorporate human skeleton length conservation and symmetry priors as well as temporal constraints to enhance the consistency and continuity for the estimated skeleton of a moving human body. Given an initial estimation of the skeleton joint positions provided per frame by the Kinect SDK or Nuitrack SDK, which do not follow the aforementioned priors and can prone to errors, our framework improves the accuracy of these pose estimates based on the length and symmetry constraints. In addition, our method is device-independent and can be integrated into skeleton extraction SDKs for refinement, allowing the detection of outliers within the initial joint location estimates and predicting new joint location estimates following the temporal observations. The experimental results demonstrate the effectiveness and robustness of our approach in several cases.

Mitral valve repair is a complex operation, in which the functionality of incompetent mitral valve is reconstructed by surgical techniques. Simulation-based surgical planning system, allowing surgeons to simulate and compare potential repair strategies, could greatly improve surgical outcomes. This paper presents a practical computational framework, combining the Total Lagrangian Explicit Dynamics Finite Element Method (TLED FEM) and Smoothed Particle Hydrodynamics (SPH), to solve the interaction problem of blood and immersed mitral valves. With this completed pipeline, we can not only predict the mechanical behavior of mitral valve, but also analyze the transvalvular pressures distributed on valve leaflets. The experimental results demonstrate that our method has the potential to be applied in surgical planning simulator of mitral valve repair.

This paper presents a novel augmented reality (AR) interactive environment for neurosurgical training. Comparing with traditional virtual reality based neurosurgical simulator, our system provides a more natural and intuitive fashion for surgeons. To achieve holographic visualization of virtual brain on 3D-printed skull (workspace), the first step is to reconstruct the personalized anatomy structure from segmented MR imaging. Then, tailored to the computational power of HoloLens, we employ the mass-spring method to model the mechanical response of brain. After that, a precise registration method is employed to map the virtual-real spatial information, which can overlay the virtual operative brain on workspace. In addition, bimanual haptic interface is also integrated into our simulator, which is more similar with real neurosurgery. In experiments, we conduct accuracy validation on our registration method, as well as the validity test on the developed simulators. The results demonstrate that our simulator can provide high-accuracy augmented visualization effects and deep immersion for novice surgeons.

Virtual surgery is a serious game which provides an opportunity to acquire cognitive and technical surgical skills via virtual surgical training and planning. However, interactively and realistically manipulating the human organ and simulating its motion under interaction is still a challenging task in this field. The underlying reason for this issue is the conflict requirements for physical constraints with high fidelity and real-time performance. To achieve realistic simulation of human organ motion with volume conservation, smooth interpolation under large deformation and precise frictional contact mechanics of global behavior in surgical scenario. This paper presents a novel and effective patch Green coordinates based interpolation for embedded deformable model to achieve the volume-preserving and smooth interpolation effects. Besides, we resolve the frictional contact mechanics for embedded deformable model, and further provide the precise boundary conditions for mechanical solver. In addition, our embedded deformable model is based on the total lagrangian explicit dynamics (TLED) finite element method (FEM) solver, which can well handle the large biological tissue deformation with both nonlinear geometric and material properties. In real compression experiments, our method can achieve liver deformation with average accuracy of 3.02 mm. Besides, the experimental results demonstrate that our method can also achieve smoother interpolation and volume-preserving effects than original embedded deformable model, and allows complex and accurate organ motion with mechanical interactions in virtual surgery.

Turbulent wake is crucial for the visually appealing effects of liquid. Unfortunately, it is challenging to realistically simulate this phenomenon with ring-shaped vortical structures. To tackle this issue, we propose a filament-based turbulent wake synthesis method for realistically simulating the turbulent wake with ring-shaped vortical structures. The filaments are sampled at the separation points on the obstacle surface and emitted into the liquid flow to generate structured turbulent wake. Besides, the surface tension model is incorporated to generate natural turbulent wake diffusion visual effects in liquid by the anticurvature effects. The proposed approach can realistically and effectively synthesize the turbulent wake with ring-shaped vortical structures and make it diffuse naturally. The experimental results demonstrate that our method outperforms than the vortex particle-based method in synthesizing appealing turbulent wake.

Background Biomechanical deformable volumetric registration can help improve safety of surgical interventions by ensuring the operations are extremely precise. However, this technique has been limited by the accuracy and the computational efficiency of patient-specific modeling. Methods This study presents a tissue–tissue coupling strategy based on penalty method to model the heterogeneous behavior of deformable body, and estimate the personalized tissue–tissue coupling parameters in a data-driven way. Moreover, considering that the computational efficiency of biomechanical model is highly dependent on the mechanical resolution, a practical coarse-to-fine scheme is proposed to increase runtime efficiency. Particularly, a detail enrichment database is established in an offline fashion to represent the mapping relationship between the deformation results of high-resolution hexahedral mesh extracted from the raw medical data and a newly constructed low-resolution hexahedral mesh. At runtime, the mechanical behavior of human organ under interactions is simulated with this low-resolution hexahedral mesh, then the microstructures are synthesized in virtue of the detail enrichment database. Results The proposed method is validated by volumetric registration in an abdominal phantom compression experiments. Our personalized heterogeneous deformable model can well describe the coupling effects between different tissues of the phantom. Compared with high-resolution heterogeneous deformable model, the low-resolution deformable model with our detail enrichment database can achieve 9.4× faster, and the average target registration error is 3.42 mm, which demonstrates that the proposed method shows better volumetric registration performance than state-of-the-art. Conclusions Our framework can well balance the precision and efficiency, and has great potential to be adopted in the practical augmented reality image-guided robotic systems.