Search

Non-rigid point cloud registration holds significant importance for human body pose analysis in the fields of sports, medicine, gaming, etc. In this paper, we propose a non-rigid point cloud registration algorithm based on geodesic distance measurement, which can improve the accuracy of the registration for matching point pairs during non-rigid deformations. Firstly, a graph is constructed for two sets of point clouds using geodesic distance measurement considering that geodesic distance changes minimally during non-rigid deformation of the human body, which can preserve the point cloud matching information between corresponding points. Furthermore, a maximal clique search is employed to find combinations of matching pairs between point clouds. Finally, by driving the human body model parameters, sparse matching pairs are overlapped as much as possible to achieve non-rigid point cloud registration of the human body. The accuracy of the proposed algorithm is verified with FAUST and CAPE datasets. [ABSTRACT FROM AUTHOR]

Non-rigid point set registration is essential for surgical navigation systems, for example in image-guided liver surgery, where accurate alignment of preoperative liver point set with intraoperative liver point set is crucial. However, accurate registration faces several challenges such as intraoperative data noise and the limited overlap between preoperative model and intraoperative data. In order to find non-rigid registration methods that are better able to cope with these challenges, we have conducted a comparative analysis of several open-sourced non-rigid registration methods, evaluating their robustness and accuracy under varying noise levels and overlapping ratios. The root-mean-square error (RMSE) is utilized to measure the accuracy of registration between the warped preoperative and intraoperative liver point sets, comparing one optimization-based and four data-driven registration approaches. Experiments mimicked clinical scenarios with controlled noise magnitudes and overlap ratios. The correspondences-based method Scoop outperforms other methods under conditions with low overlap ratio being 50% and 25%, as well as all noise magnitudes, consistently demonstrating lower registration errors than other methods(p < 0.001). The superior performance of Scoop indicates its potential as a reliable solution for accurate image-to-patient liver registration in surgical navigation. In addition, we find that the correspondences-based method performs better than the correspondences-free method. We believe this study provides valuable comparative results and insight, which will help researchers and clinicians in selecting optimal registration methods for liver surgery. [ABSTRACT FROM AUTHOR]

Producing spatial transformations that are diffeomorphic is a key goal in deformable image registration. As a diffeomorphic transformation should have positive Jacobian determinant | J | everywhere, the number of pixels (2D) or voxels (3D) with | J | < 0 has been used to test for diffeomorphism and also to measure the irregularity of the transformation. For digital transformations, | J | is commonly approximated using a central difference, but this strategy can yield positive | J | 's for transformations that are clearly not diffeomorphic—even at the pixel or voxel resolution level. To show this, we first investigate the geometric meaning of different finite difference approximations of | J | . We show that to determine if a deformation is diffeomorphic for digital images, the use of any individual finite difference approximation of | J | is insufficient. We further demonstrate that for a 2D transformation, four unique finite difference approximations of | J | 's must be positive to ensure that the entire domain is invertible and free of folding at the pixel level. For a 3D transformation, ten unique finite differences approximations of | J | 's are required to be positive. Our proposed digital diffeomorphism criteria solves several errors inherent in the central difference approximation of | J | and accurately detects non-diffeomorphic digital transformations. The source code of this work is available at https://github.com/yihao6/digital_diffeomorphism. [ABSTRACT FROM AUTHOR]

Background: In laparoscopic liver surgery, accurately predicting the displacement of key intrahepatic anatomical structures is crucial for informing the doctor's intraoperative decision‐making. However, due to the constrained surgical perspective, only a partial surface of the liver is typically visible. Consequently, the utilization of non‐rigid volume to surface registration methods becomes essential. But traditional registration methods lack the necessary accuracy and cannot meet real‐time requirements. Purpose: To achieve high‐precision liver registration with only partial surface information and estimate the displacement of internal liver tissues in real‐time. Methods: We propose a novel neural network architecture tailored for real‐time non‐rigid liver volume to surface registration. The network utilizes a voxel‐based method, integrating sparse convolution with the newly proposed points of interest (POI) linear attention module. POI linear attention module specifically calculates attention on the previously extracted POI. Additionally, we identified the most suitable normalization method RMSINorm. Results: We evaluated our proposed network and other networks on a dataset generated from real liver models and two real datasets. Our method achieves an average error of 4.23 mm and a mean frame rate of 65.4 fps in the generation dataset. It also achieves an average error of 8.29 mm in the human breathing motion dataset. Conclusions: Our network outperforms CNN‐based networks and other attention networks in terms of accuracy and inference speed. [ABSTRACT FROM AUTHOR]

Nowadays, 3D human models are widely used in the garment industry where it is important to reconstruct compliant human model from scan of the body under partial dress for privacy reasons. A new 3D human construction method based on the combination of parametric and nonparametric reconstruction is proposed here. Inputs to the method include the raw scan and partial anthropometric parameters. The scan is divided into exposed area and clothing occluded area for modeling separately. The information of the clothing occluded area is restored by semantic parametric human modeling and pose fitting. The information of the exposed area is restored by the method of non-rigid registration. Then, the two types of information are fused to reconstruct the final human model. The experiment divided the human scan into three common situations: naked, partially clothed and highly clothed body scanning. The results of the qualitative and quantitative analyses show that the method is able to fit the parametric human model to the exposed area scan while matching the user input on the anthropometric parameters in the clothing occluded area. It is worth pointing out that the connection between the two areas is smooth. The performance is also better than previous related methods. The proposed method reduces the dressing requirements for reconstructing the human body from 3D scan and also demonstrates the validity, accuracy and versatility of the method for reconstructing human model. [ABSTRACT FROM AUTHOR]

Augmented Reality (AR) holds the potential to revolutionize surgical procedures by allowing surgeons to visualize critical structures within the patient’s body. This is achieved through superimposing preoperative organ models onto the actual anatomy. Challenges arise from dynamic deformations of organs during surgery, making preoperative models inadequate for faithfully representing intraoperative anatomy. To enable reliable navigation in augmented surgery, modeling of intraoperative deformation to obtain an accurate alignment of the preoperative organ model with the intraoperative anatomy is indispensable. Despite the existence of various methods proposed to model intraoperative organ deformation, there are still few literature reviews that systematically categorize and summarize these approaches. This review aims to fill this gap by providing a comprehensive and technical-oriented overview of modeling methods for intraoperative organ deformation in augmented reality in surgery. Through a systematic search and screening process, 112 closely relevant papers were included in this review. By presenting the current status of organ deformation modeling methods and their clinical applications, this review seeks to enhance the understanding of organ deformation modeling in AR-guided surgery, and discuss the potential topics for future advancements.

Due to the potential issue of non-rigid deformations in the manufacturing process of mobile phone frames, most existing dispensing automation methods struggled to acquire accurate dispensing positions. For this purpose, an innovative approach of dispensing trajectory generation was proposed, which addressed the limitations associated with previous methods. Firstly, the phone frames were scanned using a 3D laser profiler, then a point cloud registration algorithm based on the pose graph was introduced to reconstruct the point clouds of frames. The template-based approach was employed, and the dispensing trajectory for template was manually annotated. Subsequently, the template was aligned with the target by a non-rigid registration algorithm based on the deformation graph, thereby the transformed trajectory was obtained. Furthermore, a trajectory refinement strategy was presented to generate the robust, accurate dispensing trajectory of the target. Extensive experiments demonstrate that the errors of the generated dispensing trajectories were consistently below 0.01 mm, and the speed can meet the real-time requirement of practical applications.

4D-scans of dynamic deformable human body parts help researchers have a better understanding of spatiotemporal features. However, reconstructing 4D-scans utilizing multiple asynchronous cameras encounters two main challenges: 1) finding dynamic correspondences among different frames captured by each camera at the timestamps of the camera in terms of dynamic feature recognition, and 2) reconstructing 3D-shapes from the combined point clouds captured by different cameras at asynchronous timestamps in terms of multi-view fusion. Here, we introduce a generic framework able to 1) find and align dynamic features in the 3D-scans captured by each camera using the nonrigid-iterative-closest-farthest-points algorithm; 2) synchronize scans captured by asynchronous cameras through a novel ADGC-LSTM-based-network capable of aligning 3D-scans captured by different cameras to the timeline of a specific camera; and 3) register a high-quality template to synchronized scans at each timestamp to form a high-quality 3D-mesh model using a non-rigid registration method. With a newly developed 4D-foot-scanner, we validate the framework and create the first open-access data-set, namely the 4D-feet. It includes 4D-shapes (15 fps) of the right and left feet of 58 participants (116 feet including 5147 3D-frames), covering significant phases of the gait cycle. The results demonstrate the effectiveness of the proposed framework, especially in synchronizing asynchronous 4D-scans.

Non-rigid point cloud registration holds significant importance for human body pose analysis in the fields of sports, medicine, gaming, etc. In this paper, we propose a non-rigid point cloud registration algorithm based on geodesic distance measurement, which can improve the accuracy of the registration for matching point pairs during non-rigid deformations. Firstly, a graph is constructed for two sets of point clouds using geodesic distance measurement considering that geodesic distance changes minimally during non-rigid deformation of the human body, which can preserve the point cloud matching information between corresponding points. Furthermore, a maximal clique search is employed to find combinations of matching pairs between point clouds. Finally, by driving the human body model parameters, sparse matching pairs are overlapped as much as possible to achieve non-rigid point cloud registration of the human body. The accuracy of the proposed algorithm is verified with FAUST and CAPE datasets.

Registration of deformable objects is a fundamental prerequisite for many modern virtual reality and computer vision applications. However, due to the difficulties of acquiring labeled datasets and the inherent irregular deformation, non-rigid registration for 3D scanner-captured data remains challenging. This paper proposes an unsupervised non-rigid 3D point cloud registration network based on the self-attention mechanism. Specifically, considering the registration as the result of point drifts between the source and target shapes, a Transformer-based encoder-decoder module is utilized to estimate the point displacements. Additionally, a symmetric registration procedure is adopted with regularization loss to manage the regular deformation of points, ultimately producing reasonable registration results for real-world deformable objects. Experiments are conducted on public and synthesized datasets which simulate diversiform non-rigid 2D or 3D deformations. Numerical and qualitative experimental results demonstrate that the proposed network achieves outstanding performance and is robust in scenes with multiple interferences. [ABSTRACT FROM AUTHOR]

Objective. This study aims to develop and assess a tumor contraction model, enhancing the precision of ablative margin (AM) evaluation after microwave ablation (MWA) treatment for hepatocellular carcinomas (HCCs). Approach. We utilize a probabilistic method called the coherent point drift algorithm to align pre-and post-ablation MRI images. Subsequently, a nonlinear regression method quantifies local tumor contraction induced by MWA, utilizing data from 47 HCC with viable ablated tumors in post-ablation MRI. After automatic non-rigid registration, correction for tumor contraction involves contracting the 3D contour of the warped tumor towards its center in all orientations. Main results. We evaluate the performance of our proposed method on 30 HCC patients who underwent MWA. The Dice similarity coefficient between the post-ablation liver and the warped pre-ablation livers is found to be 0.95 ± 0.01, with a mean corresponding distance between the corresponding landmarks measured at 3.25 ± 0.62 mm. Additionally, we conduct a comparative analysis of clinical outcomes assessed through MRI over a 3 month follow-up period, noting that the AM, as evaluated by our proposed method, accurately detects residual tumor after MWA. Significance. Our proposed method showcases a high level of accuracy in MRI liver registration and AM assessment following ablation treatment. It introduces a potentially approach for predicting incomplete ablations and gauging treatment success. [ABSTRACT FROM AUTHOR]

Copyright of Chinese Journal of Intelligent Science & Technology is the property of Beijing Xintong Media Co., Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Complex distortions on calcium imaging often impair image registration accuracy. Here, we developed a registration algorithm, PatchWarp, to robustly correct slow image distortion for calcium imaging data. PatchWarp is a two-step algorithm with rigid and non-rigid image registrations. To correct non-uniform image distortions, it splits the imaging field and estimates the best affine transformation matrix for each of the subfields. The distortion-corrected subfields are stitched together like a patchwork to reconstruct the distortion-corrected imaging field. We show that PatchWarp robustly corrects image distortions of calcium imaging data collected from various cortical areas through glass window or gradient-index (GRIN) lens with a higher accuracy than existing non-rigid algorithms. Furthermore, it provides a fully automated method of registering images from different imaging sessions for longitudinal neural activity analyses. PatchWarp improves the quality of neural activity analyses and is useful as a general approach to correct image distortions in a wide range of disciplines.

Accurate registration of three-dimensional (3D) craniofacial data is fundamental work for craniofacial reconstruction and analysis. The complex topology and low-quality 3D models make craniofacial registration challenging in the iterative optimization process. In this paper, we proposed a craniofacial registration network (CR-Net) that can automatically learn the registration parameters of the non-rigid thin plate spline (TPS) transformation from the training data sets and perform the required geometric transformations to align craniofacial point clouds. The proposed CR-Net employs an improved point cloud encoder architecture, a specially designed attention mechanism that can perceive the geometric structure of the point cloud. In order to align the source and target data, Wasserstein distance loss is introduced to combined with Chamfer loss and Gaussian Mixture Models (GMM) loss as an unsupervised loss function dedicated to improves registration accuracy. After efficient training, the network can automatically generate the transformation parameters for registration, transforming the reference craniofacial data to the target craniofacial data without manual calibration of feature points or performing an iterative optimization process. Experimental results show that our method has high registration accuracy and is robust to low-quality models. [Display omitted] • A neural network for robust craniofacial point cloud registration. • Geometric attention mechanism to perceive the geometric structure of point clouds. • Introducing Wasserstein distance loss constrain unsupervised training. [ABSTRACT FROM AUTHOR]

3D object matching and registration on point clouds are widely used in computer vision. However, most existing point cloud registration methods have limitations in handling non‐rigid point sets or topology changes (e.g. connections and separations). As a result, critical characteristics such as large inter‐frame motions of the point clouds may not be accurately captured. This paper proposes a statistical algorithm for non‐rigid point sets registration, addressing the challenge of handling topology changes without the need to estimate correspondence. The algorithm uses a novel Break and Splice framework to treat the non‐rigid registration challenges as a reproduction process and a Dirichlet Process Gaussian Mixture Model (DPGMM) to cluster a pair of point sets. Labels are assigned to the source point set with an iterative classification procedure, and the source is registered to the target with the same labels using the Bayesian Coherent Point Drift (BCPD) method. The results demonstrate that the proposed approach achieves lower registration errors and efficiently registers point sets undergoing topology changes and large inter‐frame motions. The proposed approach is evaluated on several data sets using various qualitative and quantitative metrics. The results demonstrate that the Break and Splice framework outperforms state‐of‐the‐art methods, achieving an average error reduction of about 60% and a registration time reduction of about 57.8%. [ABSTRACT FROM AUTHOR]

Dense vertex-to-vertex correspondence (i.e. registration) between 3D faces is a fundamental and challenging issue for 3D &2D face analysis. While the sparse landmarks are definite with anatomically ground-truth correspondence, the dense vertex correspondences on most facial regions are unknown. In this view, the current methods commonly result in reasonable but diverse solutions, which deviate from the optimum to the dense registration problem. In this paper, we revisit dense registration by a dimension-degraded problem, i.e. proportional segmentation of a line, and employ an iterative dividing and diffusing method to reach an optimum solution that is robust to different initializations. We formulate a local registration problem for dividing and a linear least-square problem for diffusing, with constraints on fixed features on a 3D facial surface. We further propose a multi-resolution algorithm to accelerate the computational process. The proposed method is linked to a novel local scaling metric, where we illustrate the physical significance as smooth adaptions for local cells of 3D facial shapes. Extensive experiments on public datasets demonstrate the effectiveness of the proposed method in various aspects. Generally, the proposed method leads to not only significantly better representations of 3D facial data, but also coherent local deformations with elegant grid architecture for fine-grained registrations. [ABSTRACT FROM AUTHOR]

Statistical Shape Models (SSMs) are well-established tools for assessing the variability of 3D geometry and for broadening a limited set of shapes. They are widely used in medical imaging due to their ability to model complex geometries and their high efficiency as generative models. The principal step behind these techniques is a registration phase, which, in the case of complex geometries, can be a critical issue due to the correspondence problem, as it necessitates the development of correspondence mapping between shapes. The thoracic aorta, with its high level of morphological complexity, poses a multi-scale deformation problem due to the presence of several branch vessels with varying diameters. Moreover, branch vessels exhibit significant variability in shape, making the correspondence optimization even more challenging. Consequently, existing studies have focused on developing SSMs based only on the main body of the aorta, excluding the supra-aortic vessels from the analysis. In this work, we present a novel non-rigid registration algorithm based on optimizing a differentiable distance function through a modified gradient descent approach. This strategy enables the inclusion of custom, domain-specific constraints in the objective function, which act as landmarks during the registration phase. The algorithm's registration performance was tested and compared to an alternative Statistical Shape modeling framework, and subsequently used for the development of a comprehensive SSM of the thoracic aorta, including the supraaortic vessels. The developed SSM was further evaluated against the alternative framework in terms of generalisation, specificity, and compactness to assess its effectiveness. [ABSTRACT FROM AUTHOR]

Non-rigid registration of point clouds is still far from stable, especially for the largely deformed one. Sparse initial correspondences are often adopted to facilitate the process. However, there are few studies on how to build them automatically. Therefore, in this paper, we propose a robust method to compute such priors automatically, where a global and local combined strategy is adopted. These priors in different degrees of deformation are obtained by the locally geometrical-consistent point matches from the globally structural-consistent region correspondences. To further utilize the matches, this paper also proposes a novel registration method based on the Coherent Point Drift framework. This method takes both the spatial proximity and local structural consistency of the priors as supervision of the registration process and thus obtains a robust alignment for clouds with significantly different deformations. Qualitative and quantitative experiments demonstrate the advantages of the proposed method. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer-Aided Design & Computer Graphics / Jisuanji Fuzhu Sheji Yu Tuxingxue Xuebao is the property of Gai Kan Bian Wei Hui and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Abstract Optical flow estimation in human facial video, which provides 2D correspondences between adjacent frames, is a fundamental pre-processing step for many applications, like facial expression capture and recognition. However, it is quite challenging as human facial images contain large areas of similar textures, rich expressions, and large rotations. These characteristics also result in the scarcity of large, annotated real-world datasets. We propose a robust and accurate method to learn facial optical flow in a self-supervised manner. Specifically, we utilize various shape priors, including face depth, landmarks, and parsing, to guide the self-supervised learning task via a differentiable nonrigid registration framework. Extensive experiments demonstrate that our method achieves remarkable improvements for facial optical flow estimation in the presence of significant expressions and large rotations.

Purpose: In laparoscopic liver surgery, preoperative information can be overlaid onto the intra-operative scene by registering a 3D preoperative model to the intra-operative partial surface reconstructed from the laparoscopic video. To assist with this task, we explore the use of learning-based feature descriptors, which, to our best knowledge, have not been explored for use in laparoscopic liver registration. Furthermore, a dataset to train and evaluate the use of learning-based descriptors does not exist. Methods: We present the LiverMatch dataset consisting of 16 preoperative models and their simulated intra-operative 3D surfaces. We also propose the LiverMatch network designed for this task, which outputs per-point feature descriptors, visibility scores, and matched points. Results: We compare the proposed LiverMatch network with a network closest to LiverMatch and a histogram-based 3D descriptor on the testing split of the LiverMatch dataset, which includes two unseen preoperative models and 1400 intra-operative surfaces. Results suggest that our LiverMatch network can predict more accurate and dense matches than the other two methods and can be seamlessly integrated with a RANSAC-ICP-based registration algorithm to achieve an accurate initial alignment. Conclusion: The use of learning-based feature descriptors in laparoscopic liver registration (LLR) is promising, as it can help achieve an accurate initial rigid alignment, which, in turn, serves as an initialization for subsequent non-rigid registration. [ABSTRACT FROM AUTHOR]

In recent years, 3D data has been widely used in archaeology and in the field of conservation and restoration of cultural properties. Virtual restoration, which reconstructs the original state in virtual space, is one of the promising applications utilizing 3D scanning data. Though many studies of virtual restoration have been conducted, it is still challenging to restore the cultural properties that consist of multiple deformable components because it is not feasible to identify the original shape uniquely. As a solution to such a problem, we proposed a non-rigid 3D shape assembly method for virtually restoring wooden ships that are composed of multiple timbers. The deformed timber can be well represented by ruled surface. We proposed a free-form deformation method with a ruled surface and an assembly method to align the deformable components mutually. The method employs a bottom-up approach that does not require reference data for target objects. The proposed framework narrows down the searching space for optimization using the physical constraints of wooden materials, and it can also obtain optimal solutions. We also showed an experimental result, where we virtually restored King Khufu's first solar boat. The boat was originally constructed by assembling several timbers. The boat was reconstructed as a real object and is currently exhibited at a museum. However, unfortunately, the entire shape of the boat is slightly distorted. We applied the proposed method using archaeological knowledge and then showed the virtual restoration results using the acquired 3D data of the boat's components. [ABSTRACT FROM AUTHOR]

Statistical Shape Models (SSMs) are well-established tools for assessing the variability of 3D geometry and for broadening a limited set of shapes. They are widely used in medical imaging due to their ability to model complex geometries and their high efficiency as generative models. The principal step behind these techniques is a registration phase, which, in the case of complex geometries, can be a critical issue due to the correspondence problem, as it necessitates the development of correspondence mapping between shapes. The thoracic aorta, with its high level of morphological complexity, poses a multi-scale deformation problem due to the presence of several branch vessels with varying diameters. Moreover, branch vessels exhibit significant variability in shape, making the correspondence optimization even more challenging. Consequently, existing studies have focused on developing SSMs based only on the main body of the aorta, excluding the supra-aortic vessels from the analysis. In this work, we present a novel non-rigid registration algorithm based on optimizing a differentiable distance function through a modified gradient descent approach. This strategy enables the inclusion of custom, domain-specific constraints in the objective function, which act as landmarks during the registration phase. The algorithm’s registration performance was tested and compared to an alternative Statistical Shape modeling framework, and subsequently used for the development of a comprehensive SSM of the thoracic aorta, including the supra-aortic vessels. The developed SSM was further evaluated against the alternative framework in terms of generalisation, specificity, and compactness to assess its effectiveness.

3d reconstruction of deformable objects in dynamic scenes forms the fundamental basis of many robotic applications. Existing mesh-based approaches compromise registration accuracy, and lose important details due to interpolation and smoothing. Additionally, existing non-rigid registration techniques struggle with unindexed points and disconnected manifolds. We propose a novel non-rigid registration framework for raw, unstructured, deformable point clouds purely based on geometric features. The global non-rigid deformation of an object is formulated as an aggregation of locally rigid transformations. The concept of locality is embodied in soft patches described by geometrical properties based on SHOT descriptor and its neighborhood. By considering the confidence score of pairwise association between soft patches of two scans (not necessarily consecutive), a computed similarity matrix serves as the seed to grow a correspondence graph which leverages rigidity terms defined in As-Rigid-As-Possible for pruning and optimization. Experiments on simulated and publicly available datasets demonstrate the capability of the proposed approach to cope with large deformations blended with numerous missing parts in the scan process.

(1) Background: Three-dimensional (3D) facial anatomical landmarks are the premise and foundation of facial morphology analysis. At present, there is no ideal automatic determination method for 3D facial anatomical landmarks. This research aims to realize the automatic determination of 3D facial anatomical landmarks based on the non-rigid registration algorithm developed by our research team and to evaluate its landmark localization accuracy. (2) Methods: A 3D facial scanner, Face Scan, was used to collect 3D facial data of 20 adult males without significant facial deformities. Using the radial basis function optimized non-rigid registration algorithm, TH-OCR, developed by our research team (experimental group: TH group) and the non-rigid registration algorithm, MeshMonk (control group: MM group), a 3D face template constructed in our previous research was deformed and registered to each participant's data. The automatic determination of 3D facial anatomical landmarks was realized according to the index of 32 facial anatomical landmarks determined on the 3D face template. Considering these 32 facial anatomical landmarks manually selected by experts on the 3D facial data as the gold standard, the distance between the automatically determined and the corresponding manually selected facial anatomical landmarks was calculated as the "landmark localization error" to evaluate the effect and feasibility of the automatic determination method (template method). (3) Results: The mean landmark localization error of all facial anatomical landmarks in the TH and MM groups was 2.34 ± 1.76 mm and 2.16 ± 1.97 mm, respectively. The automatic determination of the anatomical landmarks in the middle face was better than that in the upper and lower face in both groups. Further, the automatic determination of anatomical landmarks in the center of the face was better than in the marginal part. (4) Conclusions: In this study, the automatic determination of 3D facial anatomical landmarks was realized based on non-rigid registration algorithms. There is no significant difference in the automatic landmark localization accuracy between the TH-OCR algorithm and the MeshMonk algorithm, and both can meet the needs of oral clinical applications to a certain extent. [ABSTRACT FROM AUTHOR]

The goal of non-rigid point set registration is to estimate the optimal correspondence between points, and then recover the non-rigid deformation between point sets in a specific way, typically by using a set of complex interpolation functions. Many non-rigid matching algorithms have been studied, but only a few algorithms fully exploit the local structure between point sets. To improve the accuracy of point set registration, this paper proposes a new non-rigid registration algorithm that uses both the global structure and the stable local structure of a non-rigid shape to constrain the registration. Specifically, we consider the point set registration problem as a probability assignment problem, with the probability determined by the Gaussian mixture model and the local structure of the point set. In particular, the Hausdorff distance can effectively measure the similarity of the local structure of the point set in the proposed algorithm. The transformation between the two-point sets is determined by the reproducing kernel Hilbert space based on the motion coherence theory once the correspondence is determined. A significant number of experiments show that the proposed technique has higher registration accuracy than several other state-of-the-art algorithms when dealing with non-rigid registration problems, especially when the point set contains outliers and severely missing points. [ABSTRACT FROM AUTHOR]

Optical flow estimation in human facial video, which provides 2D correspondences between adjacent frames, is a fundamental pre-processing step for many applications, like facial expression capture and recognition. However, it is quite challenging as human facial images contain large areas of similar textures, rich expressions, and large rotations. These characteristics also result in the scarcity of large, annotated real-world datasets. We propose a robust and accurate method to learn facial optical flow in a self-supervised manner. Specifically, we utilize various shape priors, including face depth, landmarks, and parsing, to guide the self-supervised learning task via a differentiable nonrigid registration framework. Extensive experiments demonstrate that our method achieves remarkable improvements for facial optical flow estimation in the presence of significant expressions and large rotations. [ABSTRACT FROM AUTHOR]

In this paper, we propose an accurate and rapid non-rigid registration method between blood vessels in temporal 3D cardiac computed tomography angiography images of the same patient. This method provides auxiliary information that can be utilized in the diagnosis and treatment of coronary artery diseases. The proposed method consists of the following four steps. First, global registration is conducted through rigid registration between the 3D vessel centerlines obtained from temporal 3D cardiac CT angiography images. Second, point matching between the 3D vessel centerlines in the rigid registration results is performed, and the corresponding points are defined. Third, the outliers in the matched corresponding points are removed by using various information such as thickness and gradient of the vessels. Finally, non-rigid registration is conducted for hierarchical local transformation using an energy function. The experiment results show that the average registration error of the proposed method is 0.987 mm, and the average execution time is 2.137 s, indicating that the registration is accurate and rapid. The proposed method that enables rapid and accurate registration by using the information on blood vessel characteristics in temporal CTA images of the same patient. [ABSTRACT FROM AUTHOR]

Abstract In this study, a non-tensor product B-spline algorithm is applied to the search space of the registration process, and a new method of image non-rigid registration is proposed. The tensor product B-spline is a function defined in the two directions of x and y, while the non-tensor product B-spline S 2 1 Δ mn 2 $$ {S}_2^1\left({\Delta}_{mn}^{(2)}\right) $$ is defined in four directions on the 2-type triangulation. For certain problems, using non-tensor product B-splines to describe the non-rigid deformation of an image can more accurately extract the four-directional information of the image, thereby describing the global or local non-rigid deformation of the image in more directions. Indeed, it provides a method to solve the problem of image deformation in multiple directions. In addition, the region of interest of medical images is irregular, and usually no value exists on the boundary triangle. The value of the basis function of the non-tensor product B-spline on the boundary triangle is only 0. The algorithm process is optimized. The algorithm performs completely automatic non-rigid registration of computed tomography and magnetic resonance imaging images of patients. In particular, this study compares the performance of the proposed algorithm with the tensor product B-spline registration algorithm. The results elucidate that the proposed algorithm clearly improves the accuracy.

Breast cancer is the most common cancer in women worldwide. Screening programs and imaging improvements have increased the detection of clinically occult non-palpable lesions requiring preoperative localization. Wire guided localization (WGL) is the current standard of care for the excision of non-palpable carcinomas during breast conserving surgery. Due to the current limitations of intraoperative tumor localization approaches, the integration of multimodal imaging information may be especially relevant in surgical planning. This research proposes a novel method for performing preoperative image-to-surgical surface data alignment to determine the position of the tumor at the time of surgery and aid preoperative planning. First, the volume of the breast in the surgical position is reconstructed and a set of surface correspondences is defined. Then, the preoperative (prone) and intraoperative (supine) volumes are co-registered using landmark driven non-rigid registration methods. We compared the performances of diffeomorphic and Bspline based registration methods. Finally, our method was validated using clinical data from 67 patients considering as target registration error (TRE) the distance between the estimated tumor position and the reference surgical position. The proposed method achieved a TRE of 16.21 ± 8.18 mm and it could potentially assist the surgery planning and guidance of breast cancer treatment in the clinical practice.

Four-dimensional magnetic resonance imaging (4D-MRI) is becoming increasingly important in radiotherapy treatment planning for its ability to simultaneously provide 3D structural information and temporal profiles of the examined tissues in a non-ionizing manner. However, the relatively long acquisition time and the resulting motion artifacts severely limit the further application of 4D-MRI. In this paper, we propose a novel motion-aligned reconstruction method based on higher degree total variation and locally low-rank regularization (maHDTV-LLR) to recover 4D MR images from the highly undersampled Fourier coefficients. Specifically, we propose a two-stage reconstruction framework alternating between a motion alignment step and a regularized optimization reconstruction step. Moreover, we incorporate the 3D-HDTV and the locally low-rank penalties into a unified framework to simultaneously exploit the spatial and temporal correlation of the 4D-MRI data. A fast alternating minimization algorithm based on variable splitting is utilized to solve the optimization problem efficiently. The performance of the proposed method is demonstrated in the context of 4D cardiac and abdominal MR images reconstruction with high undersampling factors. Numerical results show that the proposed method enables accelerated 4D-MRI with improved image quality and reduced artifacts. [ABSTRACT FROM AUTHOR]

Aiming at the problem of tissue section images registration, a global non-rigid registration method based on local feature projective transformation was studied to obtain better matching effect. Firstly, the spatial domain enhancement and frequency domain enhancement were combined to preprocess the image, and the contour features around the tissue were highlighted by matching filtering. Secondly, the scale-invariant feature transform(SIFT) features of the processed image were extracted and preliminarily matched. According to the coordinates of the matched feature points, the projective transformation matrix parameters between the two images were calculated by random sample consensus(RANSAC) method, and the outliers were further removed. The matrix can be used for global registration. Thirdly, according to the feature points extracted by RANSAC method, the K-nearest neighbor method was used for image region divided, and projective registration matrix was solved for different local regions separately. Compared with the global registration matrix, the global non-rigid registration model of tissue section images was obtained. The experimental results show that for different tissue section images, the local projective transformation registration method can basically improve the registration accuracy, but for the image pairs with small local deformation, the improvement of the registration accuracy of the local projective registration is limited. [ABSTRACT FROM AUTHOR]

• Unedited propagated contours are suitable for CBCT-based adaptive radiotherapy. • Guidelines for reviewing propagated OAR contours when close to tolerance. • Analysis of propagated contours for adaptive radiotherapy in head and neck. • Contours propagated with 5 commercially available tools. Adaptive radiotherapy relies on rapid recontouring for replanning. Contour propagation offers workflow efficiencies, but the impact of using unedited propagated OAR contours directly during re-optimisation is unclear. Plans for ten head and neck patients were created on the planning CT scan. OAR contours for the spinal cord, brainstem, parotids and larynx were then propagated to five shading-corrected CBCTs equally spaced throughout treatment using five commercial packages. Two reference contours were created on the CBCTs by (1) a clinician and (2) a geometric consensus from the propagated contours. Treatment plans were re-optimised on each CBCT for each set of contours, and the DVH statistic differences to the reference contours were calculated. The spread of DVH statistic differences between the 5th and 95th percentiles was quantified. The spread of DVH statistic differences was 3.7 Gy compared to the clinician contour and 3.3 Gy compared to the consensus contour for the brainstem (and PRV) and 2.4 Gy and 2 Gy for the spinal cord (and PRV), across all 5 auto-contouring solutions. The parotids and larynx showed differences of 3.7 Gy compared to the clinician and 0.9 Gy to the consensus contour, with the larger difference for the clinician possibly caused by uncertainty in the clinician standard due to poor image quality on the CBCTs. Propagated OAR contours can be used safely for adaptive radiotherapy replanning, however, where organ doses are close to clinical tolerance then the contours should be reviewed for accuracy regardless of the propagation software used. [ABSTRACT FROM AUTHOR]

Augmented Reality (AR) holds the potential to revolutionize surgical procedures by allowing surgeons to visualize critical structures within the patient's body. This is achieved through superimposing preoperative organ models onto the actual anatomy. Challenges arise from dynamic deformations of organs during surgery, making preoperative models inadequate for faithfully representing intraoperative anatomy. To enable reliable navigation in augmented surgery, modeling of intraoperative deformation to obtain an accurate alignment of the preoperative organ model with the intraoperative anatomy is indispensable. Despite the existence of various methods proposed to model intraoperative organ deformation, there are still few literature reviews that systematically categorize and summarize these approaches. This review aims to fill this gap by providing a comprehensive and technical-oriented overview of modeling methods for intraoperative organ deformation in augmented reality in surgery. Through a systematic search and screening process, 112 closely relevant papers were included in this review. By presenting the current status of organ deformation modeling methods and their clinical applications, this review seeks to enhance the understanding of organ deformation modeling in AR-guided surgery, and discuss the potential topics for future advancements.

Purpose: Deformable image registration establishes non-linear spatial correspondences between fixed and moving images. Deep learning-based deformable registration methods have been widely studied in recent years due to their speed advantage over traditional algorithms as well as their better accuracy. Most existing deep learning-based methods require neural networks to encode location information in their feature maps and predict displacement or deformation fields through convolutional or fully connected layers from these high-dimensional feature maps. We present vector field attention (VFA), a novel framework that enhances the efficiency of the existing network design by enabling direct retrieval of location correspondences., Approach: VFA uses neural networks to extract multi-resolution feature maps from the fixed and moving images and then retrieves pixel-level correspondences based on feature similarity. The retrieval is achieved with a novel attention module without the need for learnable parameters. VFA is trained end-to-end in either a supervised or unsupervised manner., Results: We evaluated VFA for intra- and inter-modality registration and unsupervised and semi-supervised registration using public datasets as well as the Learn2Reg challenge. VFA demonstrated comparable or superior registration accuracy compared with several state-of-the-art methods., Conclusions: VFA offers a novel approach to deformable image registration by directly retrieving spatial correspondences from feature maps, leading to improved performance in registration tasks. It holds potential for broader applications., (© 2024 Society of Photo-Optical Instrumentation Engineers (SPIE).)

This study aimed to develop an automatic methodology for mandibular landmarking and measurement using non-rigid registration as well as analyze the accuracy of automatic landmarking and measurements., Statistical analysis., Digital technology center, tertiary hospital., 130 healthy Chinese adults with equal gender distribution, average age 28.2 ± 5.6 years., Four mean shape mesh templates were generated from 100 head CT scans. Following manual indication of landmarks, these templates were applied for automatic landmark annotation and measurements on mandibles from another 30 head CT scans, using non-rigid iterative closest point registration., Differences of landmark coordinates and measurements between automatic and manual annotation were analyzed using mean difference, centroid size, Euclidean distances and intraclass correlation coefficient (ICC), assessing the accuracy and validity of automatic landmark annotation., The majority of automatic landmarks (16/22) did not exhibit consistent displacement to specific direction. ICCs of all landmark coordinates exceed 0.950, with 87.9% larger than 0.990. The average Euclidean distance between manual and automatic landmarks was 2.038 ± 0.947 mm. Most ICCs of linear and angular measurements between manual and automatic annotation (20/26) exceeded 0.900, with the average errors being 1.425 ± 0.973 mm and 2.257 ± 0.649 °, respectively., A novel and efficient method for automatic landmark annotation was established based on non-rigid registration. Its credibility and accuracy in mandibular annotation and measurements were demonstrated., Competing Interests: Declaration of Conflicting InterestsThe authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Chiari I malformation is characterized by the herniation of cerebellar tonsils below the foramen magnum. It is often accompanied by syringomyelia and neurosurgical management is still controversial. In fact, it is frequent that some symptomatic patients initially undergo bony decompression of the posterior fossa and need in a short time more invasive surgery with higher morbility (e.g., decompression of posterior fossa with dural plastic, with or without tonsillar coarctation) because of unsatisfactory results at MRI controls. This study proposes a machine learning approach (based on SVM classifier), applied to different morphometric indices estimated from sagittal MRI and some information on the patient (i.e., age and symptoms at diagnosis), to recognize patients with higher risk of syringomyelia and clinical deterioration. Our database includes 58 pediatric patients who underwent surgery treatment. A negative outcome at 1 year from the intervention was observed in 38% of them (accuracy of 62%). Our algorithm allows us to increase the accuracy to about 71%, showing it to be a valid support to neurosurgeons in refining the clinical picture. [ABSTRACT FROM AUTHOR]

Purpose: We present a novel augmented reality (AR) surgical navigation method with ultrasound-assisted point cloud registration for percutaneous ablation of liver tumors. A preliminary study is carried out to verify its feasibility. Methods: Two three-dimensional (3D) point clouds of the liver surface are derived from the preoperative images and intraoperative tracked US images, respectively. To compensate for the soft tissue deformation, the point cloud registration between the preoperative images and the liver is performed using the non-rigid iterative closest point (ICP) algorithm. A 3D AR device based on integral videography technology is designed to accurately display naked-eye 3D images for surgical navigation. Based on the above registration, naked-eye 3D images of the liver surface, planning path, entry points, and tumor can be overlaid in situ through our 3D AR device. Finally, the AR-guided targeting accuracy is evaluated through entry point positioning. Results: Experiments on both the liver phantom and in vitro pork liver were conducted. Several entry points on the liver surface were used to evaluate the targeting accuracy. The preliminary validation on the liver phantom showed average entry-point errors (EPEs) of 2.34 ± 0.45 mm, 2.25 ± 0.72 mm, 2.71 ± 0.82 mm, and 2.50 ± 1.11 mm at distinct US point cloud coverage rates of 100%, 75%, 50%, and 25%, respectively. The average EPEs of the deformed pork liver were 4.49 ± 1.88 mm and 5.02 ± 2.03 mm at the coverage rates of 100% and 75%, and the average covered-entry-point errors (CEPEs) were 4.96 ± 2.05 mm and 2.97 ± 1.37 mm at 50% and 25%, respectively. Conclusion: Experimental outcomes demonstrate that the proposed AR navigation method based on US-assisted point cloud registration has achieved an acceptable targeting accuracy on the liver surface even in the case of liver deformation. [ABSTRACT FROM AUTHOR]