Search

We present a Non-Rigid Structure-from-Motion (NRSfM) method to reconstruct an object whose topology is known. We represent the topology by a 3D shape that weakly resembles the object, which we call a Topological Template (ToTem). The ToTem has two main differences with the template used in Shape-from-Template (SfT). First, the shape in the ToTem is not necessarily feasible for the object, whereas it must be in the SfT template. Second, the ToTem only models shape, excluding the classical texture-map representing colour in the SfT template. These two differences greatly alleviate the practical difficulty of constructing a template. However, they make the reconstruction problem challenging, as they preclude the use of strong deformation constraints between the template shape and the reconstruction and the possibility of directly establishing correspondences between the template and the images. Our method uses an isometric deformation prior and proceeds in four steps. First, it reconstructs point-clouds from the images. Second, it aligns the ToTem to the point-clouds. Third, it creates a coherent surface parameterisation. Fourth, it performs a global refinement, posed as Bundle Adjustment (BA). We show experimentally that our method outperforms the existing methods for its isolated steps and NRSfM methods overall, in terms of 3D accuracy, ability to reconstruct the object's visible surface and ability to approximate the object's invisible surface. [ABSTRACT FROM AUTHOR]

Most reconstruction algorithms for non-rigid three-dimensional (3D) images assume that non-rigidity can be represented as a linear combination of a fixed number of rigid bases. However, this assumption struggles to establish reliable shape functions and initial values for nonlinear and non-rigid motions, decreasing reconstruction reliability. This paper introduces an enhanced-reliability reconstruction algorithm for non-rigid 3D images. Our algorithm models the dynamic non-rigid shape basis as a low-rank matrix composed of image points and depth factors, improving the restoration of non-rigid shape base changes and providing accurate parameters for constructing objective functions. By leveraging manifold alignment and physical continuity constraints, our method optimizes the function structures. Assuming minimal reconstruction error and shape change, we solve for the motion structure parameters and select the key initial shape basis value by minimizing the objective function with the L-M nonlinear optimization method. Our experimental results on 3D image sequence reconstructions demonstrate significant error reduction, underscoring our model’s credibility, robust reliability, and minimal re-projection error.

Over the past decades, robotic automation has expanded in numerous industrial sectors; however, manufacturing operations involving the manipulation of non-rigid products are mostly preserved manual. The dynamic distortion of flexible objects underlines limitations in robot cognition and dexterity. Inspired by the gaps in composites industry automation, this paper presents a novel multifunctional robot end-effector for the robotic automation of composites layup. Aiming a holistic confrontation of layup challenges, the proposed end-effector incorporates tools for (a) the manipulation of sheet materials, like composite fabrics of variable dimensions; (b) the grasping of core materials, like foam blocks; (c) the handling of peripheral tools; (d) the exertion of fitting forces; and (e) the application of resin in diverse mold geometry cavities. The meticulous design of the end-effector's configuration allows for consistent and collision-free tool usage with no excess robot tool changes, granting even higher productivity and efficiency. The enhancement in robot dexterity and proficiency for layup operations is validated within an automotive case, where semi-automation is intended for the improvement in overall workstation's efficiency, quality, ergonomics, and well-being. [ABSTRACT FROM AUTHOR]

Aiming to improve the performance of non-rigid point set registration, this paper proposes a probabilistic method with student's t mixture model (SMM) under the Bayesian inference framework. In the proposed method, non-rigid point set registration is formulated as a probabilistic density estimation problem with SMM in Bayesian manner. In order to improve the robustness to rotation degradation, we consider the rotation transformation in modeling non-rigid displacement. Then, the hierarchical Bayesian model of non-rigid point set registration is constructed, and approximate posteriors of model parameters are derived by the variational Bayesian Expectation Maximization update rules, which can provide the uncertainty measurements of parameters. For those parameters without priors imposed, the updating formulae are obtained by directly maximizing variational lower bound. Finally, an empirical coarse-to-fine algorithm is designed to perform non-rigid point set registration process. The experimental results demonstrate that the proposed method can achieve higher matching performance compared with other several state-of-the-art registration methods. [ABSTRACT FROM AUTHOR]

Occlusion, rotation and other factors affect human motion structure because of the incomplete acquired image sequence, resulting in poor performance of non-rigid three-dimensional (3D) motion pose reconstruction. A non-rigid 3D reconstruction and high-precision correction method for motion pose are studied in this paper. A non-rigid imaging model is designed to obtain 3D moving images. According to the frame difference and morphological processing, the background of image is separated and denoised. Combined with motion analysis, 3D motion pose features are extracted as identification of non-rigid 3D motion error actions in a hybrid Convolution Neural Network-Hidden Markov Model to train the correction coefficients, which are used to adjust the pose in 3D motion reconstruction and realise correction. Experimental results show that this method has high precision reconstruction and correction of non-rigid 3D motion pose. [ABSTRACT FROM AUTHOR]

We propose LaplacianFusion, a novel approach that reconstructs detailed and controllable 3D clothed-human body shapes from an input depth or 3D point cloud sequence. The key idea of our approach is to use Laplacian coordinates, well-known differential coordinates that have been used for mesh editing, for representing the local structures contained in the input scans, instead of implicit 3D functions or vertex displacements used previously. Our approach reconstructs a controllable base mesh using SMPL, and learns a surface function that predicts Laplacian coordinates representing surface details on the base mesh. For a given pose, we first build and subdivide a base mesh, which is a deformed SMPL template, and then estimate Laplacian coordinates for the mesh vertices using the surface function. The final reconstruction for the pose is obtained by integrating the estimated Laplacian coordinates as a whole. Experimental results show that our approach based on Laplacian coordinates successfully reconstructs more visually pleasing shape details than previous methods. The approach also enables various surface detail manipulations, such as detail transfer and enhancement. [ABSTRACT FROM AUTHOR]

• A new metric, EGDTA, was developed and tested for automated DIR quality evaluation. • The EGDTA map and imposed DVF on phantom and CT images showed good agreement. • The EGDTA metric provides the expected dependence on DIR type and ROI location. • The EGDTA metric shows potential as an automated means of comparing DIR algorithms. To develop a new registration quality metric, based on the distance between image edges, for automated evaluation and comparison of DIR algorithms. Canny filter is used to create binary gradient images from input images to be compared. A small subregion of one binary image is translated relative to the other. The translational distance maximizing overlap of edges in the subregion is the local edge gradient distance to agreement (EGDTA); repeating over all subregions provides an EGDTA map. The method was tested on phantom and pelvic CT images, by applying a known deformable vector field (DVF). The method was then applied to evaluate two DIR algorithms (SICLE and Demons) for pelvic CTs from five patients. Three SICLE variants were used: Grayscale-driven (G), Contour-driven (C), and Grayscale + Contour-driven (G + C). For each patient, a planning CT was registered to three repeat CTs using the above DIR algorithms. Mean EGDTA values in concentric ring regions of interest close to and far away from the pelvic organ contours were compared. EGDTA maps and imposed DVF deformations on phantom and CT images demonstrated agreement. In comparison of the three variants of SICLE: C had lower EGDTA values close to the pelvic organs, while G showed much better performance in the regions distant from the organs compared to C; and G + C registration exhibited the lowest or comparable EGDTA value among three variants. Demons achieved the lowest EGDTA values. The EGDTA metric shows potential as an automated means of evaluating and comparing DIR algorithms. [ABSTRACT FROM AUTHOR]

Abstract Background Cardiovascular magnetic resonance T1ρ mapping may detect myocardial injuries without exogenous contrast agent. However, multiple co-registered acquisitions are required, and the lack of robust motion correction limits its clinical translation. We introduce a single breath-hold myocardial T1ρ mapping method that includes model-based non-rigid motion correction. Methods A single-shot electrocardiogram (ECG)-triggered balanced steady state free precession (bSSFP) 2D adiabatic T1ρ mapping sequence that collects five T1ρ-weighted (T1ρw) images with different spin lock times within a single breath-hold is proposed. To address the problem of residual respiratory motion, a unified optimization framework consisting of a joint T1ρ fitting and model-based non-rigid motion correction algorithm, insensitive to contrast change, was implemented inline for fast (~ 30 s) and direct visualization of T1ρ maps. The proposed reconstruction was optimized on an ex vivo human heart placed on a motion-controlled platform. The technique was then tested in 8 healthy subjects and validated in 30 patients with suspected myocardial injury on a 1.5T CMR scanner. The Dice similarity coefficient (DSC) and maximum perpendicular distance (MPD) were used to quantify motion and evaluate motion correction. The quality of T1ρ maps was scored. In patients, T1ρ mapping was compared to cine imaging, T2 mapping and conventional post-contrast 2D late gadolinium enhancement (LGE). T1ρ values were assessed in remote and injured areas, using LGE as reference. Results Despite breath holds, respiratory motion throughout T1ρw images was much larger in patients than in healthy subjects (5.1 ± 2.7 mm vs. 0.5 ± 0.4 mm, P

Previous studies on perceptual grouping found that people can use spatiotemporal and featural information to group spatially separated rigid objects into a unit while tracking moving objects. However, few studies have tested the role of objects' self‐motion information in perceptual grouping, although it is of great significance to the motion perception in the three‐dimensional space. In natural environments, objects always move in translation and rotation at the same time. The self‐rotation of the objects seriously destroys objects' rigidity and topology, creates conflicting movement signals and results in crowding effects. Thus, this study sought to examine the specific role played by self‐rotation information on grouping spatially separated non‐rigid objects through a modified multiple object tracking (MOT) paradigm with self‐rotating objects. Experiment 1 found that people could use self‐rotation information to group spatially separated non‐rigid objects, even though this information was deleterious for attentive tracking and irrelevant to the task requirements, and people seemed to use it strategically rather than automatically. Experiment 2 provided stronger evidence that this grouping advantage did come from the self‐rotation per se rather than surface‐level cues arising from self‐rotation (e.g. similar 2D motion signals and common shapes). Experiment 3 changed the stimuli to more natural 3D cubes to strengthen the impression of self‐rotation and again found that self‐rotation improved grouping. Finally, Experiment 4 demonstrated that grouping by self‐rotation and grouping by changing shape were statistically comparable but additive, suggesting that they were two different sources of the object information. Thus, grouping by self‐rotation mainly benefited from the perceptual differences in motion flow fields rather than in deformation. Overall, this study is the first attempt to identify self‐motion as a new feature that people can use to group objects in dynamic scenes and shed light on debates about what entities/units we group and what kinds of information about a target we process while tracking objects. [ABSTRACT FROM AUTHOR]

[Display omitted] • A non-rigid registration method based on image triangulation and piecewise affine transformation (TPAT) technologies. • TPAT registration method supports both automatic and semi-automatic operation types. • TPAT method achieves great performance in the benchmark test of non-rigid neural image registration. • TPAT method works well in registering multi-session two-photon imaging stacks. Accurate and efficient non-rigid registration is important to investigate neural mechanisms in multi-session two-photon (2p) imaging across a few days. The 2p imaging recordings from different sessions usually possess certain complex misalignment or huge data variance due to relocation errors during experimental operations or brain recovery. Most of the reported neural image registration tools were able to solve the registration problem in the same session with small deformation. However, the registration of neural images across multi-sessions remains a challenge. In this study, we report the development of a non-rigid registration method for 2p imaging in mice based on image triangulation and piecewise affine transformation (TPAT) technologies. The TPAT method supported both automatic and semi-automatic operation types, and both showed great performance in the benchmark test of non-rigid neural image registration. The proposed method constitutes a step forward in promoting and accelerating discoveries from multi-session 2p imaging research. [ABSTRACT FROM AUTHOR]

Background: Cardiovascular magnetic resonance T1ρ mapping may detect myocardial injuries without exogenous contrast agent. However, multiple co-registered acquisitions are required, and the lack of robust motion correction limits its clinical translation. We introduce a single breath-hold myocardial T1ρ mapping method that includes model-based non-rigid motion correction. Methods: A single-shot electrocardiogram (ECG)-triggered balanced steady state free precession (bSSFP) 2D adiabatic T1ρ mapping sequence that collects five T1ρ-weighted (T1ρw) images with different spin lock times within a single breath-hold is proposed. To address the problem of residual respiratory motion, a unified optimization framework consisting of a joint T1ρ fitting and model-based non-rigid motion correction algorithm, insensitive to contrast change, was implemented inline for fast (~ 30 s) and direct visualization of T1ρ maps. The proposed reconstruction was optimized on an ex vivo human heart placed on a motion-controlled platform. The technique was then tested in 8 healthy subjects and validated in 30 patients with suspected myocardial injury on a 1.5T CMR scanner. The Dice similarity coefficient (DSC) and maximum perpendicular distance (MPD) were used to quantify motion and evaluate motion correction. The quality of T1ρ maps was scored. In patients, T1ρ mapping was compared to cine imaging, T2 mapping and conventional post-contrast 2D late gadolinium enhancement (LGE). T1ρ values were assessed in remote and injured areas, using LGE as reference. Results: Despite breath holds, respiratory motion throughout T1ρw images was much larger in patients than in healthy subjects (5.1 ± 2.7 mm vs. 0.5 ± 0.4 mm, P < 0.01). In patients, the model-based non-rigid motion correction improved the alignment of T1ρw images, with higher DSC (87.7 ± 5.3% vs. 82.2 ± 7.5%, P < 0.01), and lower MPD (3.5 ± 1.9 mm vs. 5.1 ± 2.7 mm, P < 0.01). This resulted in significantly improved quality of the T1ρ maps (3.6 ± 0.6 vs. 2.1 ± 0.9, P < 0.01). Using this approach, T1ρ mapping could be used to identify LGE in patients with 93% sensitivity and 89% specificity. T1ρ values in injured (LGE positive) areas were significantly higher than in the remote myocardium (68.4 ± 7.9 ms vs. 48.8 ± 6.5 ms, P < 0.01). Conclusions: The proposed motion-corrected T1ρ mapping framework enables a quantitative characterization of myocardial injuries with relatively low sensitivity to respiratory motion. This technique may be a robust and contrast-free adjunct to LGE for gaining new insight into myocardial structural disorders. [ABSTRACT FROM AUTHOR]

The acquisition of high quality maps of gene expression in the rodent brain is of fundamental importance to the neuroscience community. The generation of such datasets relies on registering individual gene expression images to a reference volume, a task encumbered by the diversity of staining techniques employed, and by deformations and artifacts in the soft tissue. Recently, deep learning models have garnered particular interest as a viable alternative to traditional intensity-based algorithms for image registration. In this work, we propose a supervised learning model for general multimodal 2D registration tasks, trained with a perceptual similarity loss on a dataset labeled by a human expert and augmented by synthetic local deformations. We demonstrate the results of our approach on the Allen Mouse Brain Atlas (AMBA), comprising whole brain Nissl and gene expression stains. We show that our framework and design of the loss function result in accurate and smooth predictions. Our model is able to generalize to unseen gene expressions and coronal sections, outperforming traditional intensity-based approaches in aligning complex brain structures.

Classifying moving objects in video surveillance can be difficult, and it is challenging to classify hard and soft objects with high Accuracy. Here rigid and non-rigid objects are limited to vehicles and people. CNN is used for the binary classification of rigid and non-rigid objects. A deep-learning system using convolutional neural networks was trained using python and categorized according to their appearance. The classification is supplemented by the use of a data set, which contains two classes of images that are both rigid and not rigid that differ by illuminations. [ABSTRACT FROM AUTHOR]

The acquisition of high quality maps of gene expression in the rodent brain is of fundamental importance to the neuroscience community. The generation of such datasets relies on registering individual gene expression images to a reference volume, a task encumbered by the diversity of staining techniques employed, and by deformations and artifacts in the soft tissue. Recently, deep learning models have garnered particular interest as a viable alternative to traditional intensity-based algorithms for image registration. In this work, we propose a supervised learning model for general multimodal 2D registration tasks, trained with a perceptual similarity loss on a dataset labeled by a human expert and augmented by synthetic local deformations. We demonstrate the results of our approach on the Allen Mouse Brain Atlas (AMBA), comprising whole brain Nissl and gene expression stains. We show that our framework and design of the loss function result in accurate and smooth predictions. Our model is able to generalize to unseen gene expressions and coronal sections, outperforming traditional intensity-based approaches in aligning complex brain structures. [ABSTRACT FROM AUTHOR]

Feature matching, which refers to the establishment of accurate correspondence between two sets of feature points, plays an important role in the field of computer vision and remote sensing. Focusing on the characteristics of remote sensing images, this paper proposes a new algorithm for robust feature matching based on non-rigid transformation, which formulates the feature matching problem as a probabilistic model. Specifically, we first utilize the scale-invariant feature transform to establish the initial feature correspondences between an image pair, and the thin-plate spline is adopted for non-rigid transformation modeling, where a local geometric constraint is introduced to maintain local structures of neighboring feature points after the transformation. Under the Bayesian framework, we seek a maximum a posteriori solution of our model by using the expectation-maximization algorithm. In addition, without sacrificing the matching accuracy, we provide a fast implementation to reduce the computational complexity of the algorithm based on sparse approximation. We verify the performance of our method on a large number of remote sensing images, and the qualitative and quantitative results reveal its superiority over the state-of-the-art.

For the existence of outliers in non-rigid point set registration, a method based on Bayesian student's t mixture model(SMM) is proposed. Under the framework of variational Bayesian, the point set registration problem is converted to maximize the variational lower bound of log-likelihood, where the transformation parameters are found through variational inference. By prior model, the constraint over spatial regularization is incorporated into the Bayesian SMM, which can adaptively be determined for different data sets. Compared with Gaussian distribution, the student's t distribution is more robust to outliers. The experimental comparative analysis of simulated points and real images verify the effectiveness of the proposed method on the non-rigid point set registration with outliers.

We propose an original approach to absolute pose and structure-from-motion (SfM) which handles rolling shutter (RS) effects. Unlike most existing methods which either augment global shutter projection with velocity parameters or impose continuous time and motion through pose interpolation, we use local differential constraints. These are established by drawing analogies with non-rigid 3D vision techniques, namely shape-from-template and non-rigid SfM (NRSfM). The proposed idea is to interpret the images of a rigid surface acquired by a moving RS camera as those of a virtually deformed surface taken by a GS camera. These virtually deformed surfaces are first recovered by relaxing the RS constraint using SfT or NRSfM. Then we upgrade the virtually deformed surface to the actual rigid structure and compute the camera pose and ego-motion by reintroducing the RS constraint. This uses a new 3D-3D registration procedure that minimizes a cost function based on the Euclidean 3D point distance. This is more stable and physically meaningful than the reprojection error or the algebraic distance used in previous work. Experimental results obtained with synthetic and real data show that the proposed methods outperform existing ones in terms of accuracy and stability, even in the known critical configurations. [ABSTRACT FROM AUTHOR]

While dynamic scene reconstruction has made revolutionary progress from the earliest setup using a mass of static cameras in studio environment to the latest egocentric or hand-held moving camera based schemes, it is still restricted by the recording volume, user comfortability, human labor and expertise. In this paper, a novel solution is proposed through a real-time and robust dynamic fusion scheme using a single flying depth camera, denoted as FlyFusion. By proposing a novel topology compactness strategy for effectively regularizing the complex topology changes, and the Geometry And Motion Energy (GAME) metric for guiding the viewpoint optimization in the volumetric space, FlyFusion succeeds to enable intelligent viewpoint selection based on the immediate dynamic reconstruction result. The merit of FlyFusion lies in its concurrent robustness, efficiency, and adaptation in producing fused and denoised 3D geometry and motions of a moving target interacting with different non-rigid objects in a large space. [ABSTRACT FROM AUTHOR]

Machine vision has been extensively studied during the last decades enabling the implementation of vision systems in robotic applications usually involving dispersing of material or handling of rigid objects. Nevertheless, the manipulation of deformable parts remains an open challenge since it involves numerous potential configurations of the object increasing the training effort of the algorithms. To this end, a simple and low-cost solution employing a thresholding contour and depth-based detection method of flexible objects is proposed. This algorithm calculates the coordinates of a grasping point near an edge of the object along with the required orientation of the robot's gripper, leading to effective pick-and-place operations. The suggested approach has been implemented in 2 stages utilizing a 2D vision system. The testing of the method was performed in a case study coming from the household appliances manufacturing sector, where the efficient robotic grasping of a dishwasher gasket with mirror symmetrical edges was achieved successfully. [ABSTRACT FROM AUTHOR]

We present a feature-based image registration method, the stepwise image registration (SIR), with a closed-form solution. Our SIR creates an inlier pool and a candidate pool as the initialization, and then gradually enriches the inlier pool and refines the transformation. In each step, the enriched correspondence exclusively tunes the transformation coefficient within the confirmed inlier pairs, instead of updating the mapping using the complete putative set. In turn, the refined transformation prunes inconsistent mismatches to alleviate the incoming matching ambiguity. The context-aware locality measure (CALM) is designed for dissimilarity measure. The capability of the CALM can be enhanced by the progressive inlier pool enrichment. Finally, a retrieval process is performed based on the finest CALM and alignment, by which the inlier pool is maximized. Extensive experiments of enrichment evaluation, feature matching, image registration, and image retrieval demonstrate the favorable performance of our SIR against state-of-the-art methods. The code and datasets are available at https://github.com/sucv/SIR. [ABSTRACT FROM AUTHOR]

Fingerprints are unique and invariant, so they are widely used for biometric recognition. However, due to the problem of deformation in the actual sampling process, it may cause a great change in the fingerprint features. As the accuracy of fingerprint recognition depends on the quality of fingerprints, non-rigid registration algorithms are particularly important. Most existing non-rigid registration algorithms estimate the distortion only by minutiae and gratitude of fingerprints, while direction of the ridges is neglected or used indirectly. In this paper, we proposed a novel model based algorithm for non-rigid fingerprint registration using image fields. As direction information is very important for spatial transformation in registration and the image fields contain the direction of fingerprint ridges, by combing image fields with the traditional model based algorithm, we directly introduce orientation of the ridges to the model for estimating the distortion, and thus make a better use of the direction information of fingerprints and simplify the deformation model. Considering that delta/cores and minutia are sometimes hard to extract accurately and reliably, we used the whole image for matching directly. Experiments have been carried out on four representative databases, namely FVC2004 DB1, Tsinghua Distorted Fingerprint database, NIST SD27 database and NIST SD30 database. We also compared our algorithm with other state-of-the-art algorithms, the experimental results show the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

• A novel non-rigid registration method for medical images is proposed by introducing the image field into Demons algorithm. • Introduce the orientation field for registration and make use of the directional information. • There is no extraction of feature points and manual intervention in the proposed method. • Experiments on BrainWeb and FIRE databases show the effectiveness of the proposed registration method for medical images. In medical imaging application, non-rigid registration is an important step that requires both accuracy and efficiency to align the deformed query image with the reference image. For non-rigid registration, Demons is generally used as an effective algorithm. However, since the computational model of Demons uses only the gradient information, the directional information of the image has not been fully utilized. In this paper, we proposed a novel non-rigid registration method for processing medical images by introducing the image field to the Demons algorithm. As direction information is very important for spatial transformation in registration and the image fields contain the direction of the image, by combing image fields with the traditional model based registration algorithm, we introduce the orientation field directly to the model for estimating the distortion, and thus make a better use of the direction information of images and simplify the deformation model. Different performance measures are used to evaluate the qualitative measures of the proposed scheme. Experiments on brain images and fundus images show the improvement of the proposed algorithm compared to the other existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Image & Graphics is the property of Editorial Office of Journal of Image & Graphics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Highlights • Novel bias-corrected non-rigid registration approach compensating for fast and slow scan artifacts in scanning transmission electron microscopes (STEM) image series. • Bias-correction responsible for the correction of the slow scan artifacts. • Reduces fast scan noise in an image series and slow scan distortions simultaneously. • Tested on synthetic and experimental images. Abstract Aberration corrected scanning transmission electron microscopes (STEM) enable to determine local strain fields, composition and bonding states at atomic resolution. The precision to locate atomic columns is often obstructed by scan artifacts limiting the quantitative interpretation of STEM datasets. Here, a novel bias-corrected non-rigid registration approach is presented that compensates for fast and slow scan artifacts in STEM image series. The bias-correction is responsible for the correction of the slow scan artifacts and based on a explicit coupling of the deformations of the individual images in a series via a minimization of the average deformation. This allows to reduce fast scan noise in an image series and slow scan distortions simultaneously. The novel approach is tested on synthetic and experimental images and its implication on atomic resolution strain and elemental mapping is discussed. [ABSTRACT FROM AUTHOR]

Non-rigid surface-based soft tissue registration is crucial for surgical navigation systems, but its adoption still faces several challenges due to the large number of degrees of freedom and the continuously varying and complex surface structures present in the intra-operative data. By employing non-rigid registration, surgeons can integrate the pre-operative images into the intra-operative guidance environment, providing real-time visualization of the patient's complex pre- and intra-operative anatomy in a common coordinate system to improve navigation accuracy. However, many of the existing registration methods, including those for liver applications, are inaccessible to the broader community. To address this limitation, we present a comparative analysis of several open-source, non-rigid surface-based liver registration algorithms, with the overall goal of contrasting their strength and weaknesses and identifying an optimal solution. We compared the robustness of three optimization-based and one data-driven nonrigid registration algorithms in response to a reduced visibility ratio (reduced partial views of the surface) and to an increasing deformation level (mean displacement), reported as the root mean square error (RMSE) between the pre-and intra-operative liver surface meshed following registration. Our results indicate that the Gaussian Mixture Model - Finite Element Model (GMM-FEM) method consistently yields a lower post-registration error than the other three tested methods in the presence of both reduced visibility ratio and increased intra-operative surface displacement, therefore offering a potentially promising solution for pre- to intra-operative nonrigid liver surface registration.

The objective of the work is to model the shape of the sinusoidal shape of regular water waves generated in a laboratory flume. The waves are traveling in time and render a smooth surface, with no white caps or foam. Two methods are proposed, treating the water as a diffuse and specular surface, respectively. In either case, the water is presumed to take the shape of a traveling sine wave, reducing the task of the 3D reconstruction to resolve the wave parameters. The first conceived method performs the modeling part purely in 3D space. Having triangulated the points in a separate phase via bundle adjustment, a sine wave is fitted into the data in a least squares manner. The second method presents a more complete approach for the entire calculation workflow beginning in the image space. The water is perceived as a specular surface, and the traveling specularities are the only observations visible to the cameras, observations that are notably single image. The depth ambiguity is removed given additional constraints encoded within the law of reflection and the modeled parametric surface. The observation and constraint equations compose a single system of equations that is solved with the method of least squares adjustment. The devised approaches are validated against the data coming from a capacitive level sensor and on physical targets floating on the surface. The outcomes agree to a high degree.

We present a new motion tracking technique to robustly reconstruct non-rigid geometries and motions from a single view depth input recorded by a consumer depth sensor. The idea is based on the observation that most non-rigid motions (especially human-related motions) are intrinsically involved in articulate motion subspace. To take this advantage, we propose a novel $L_0$ based motion regularizer with an iterative solver that implicitly constrains local deformations with articulate structures, leading to reduced solution space and physical plausible deformations. The $L_0$ strategy is integrated into the available non-rigid motion tracking pipeline, and gradually extracts articulate joints information online with the tracking, which corrects the tracking errors in the results. The information of the articulate joints is used in the following tracking procedure to further improve the tracking accuracy and prevent tracking failures. Extensive experiments over complex human body motions with occlusions, facial and hand motions demonstrate that our approach substantially improves the robustness and accuracy in motion tracking. [ABSTRACT FROM PUBLISHER]

A series of phosphinoyldithioformate compounds were synthesized from metathesis reactions of K[R 2 P(O)CS 2 ], and R' n PbCl 4-n and R’HgCl. The compounds isolated were characterized using infrared, UV and multinuclear NMR ( 1 H, 13 C, and 31 P) spectroscopy. Structural diversity of these compounds is revealed through analysis of the infrared and NMR spectroscopy, where variable temperature 31 P NMR spectroscopy data of 6 reveals non-rigid coordination behavior and an equilibrium of aniso bonded S,S 4-membered ring, and S,O bonded 5-membered ring bonding modes in [MeHg{S 2 CP(O)(Ph) 2 }], ( 6 – 6′ ). The crystal structure of 6′ shows the mercury atom in a T-shape geometry with S,O-bidentate coordinated ligand although room temperature NMR indicated the coordination to the mercury in 6 is non rigid in solution. Variable temperature 31 P NMR spectroscopy yielded an average value of ΔG # 6-6´ = 45.7 ± 1.7 kJ mol −1 for the temperature range of 190 K–283 K. The crystal structure of [Ph 2 Pb{S 2 CP(O)(Ph) 2 } 2 (H 2 O)], 1 , reveals the lead atom in an unusual seven coordinate geometry with two S,O-bidentate coordinated ligands and axial phenyl groups. The seventh ligand is a water molecule bonded to lead forming a pentagonal plane. [ABSTRACT FROM AUTHOR]

A high resolution dense matching algorithm is presented for non-rigid image feature matching in the paper. For high resolution non-rigid images, telephoto lens is helpful in capturing fine scale features like cloth fold, pigmentation and skin pores. It brings us serious image noises which are less texture and bokeh, respectively. In order to avoid mismatch and non-uniform matching, we propose an optimized sampling method based on Gibbs dense sampling considering both texture feature similarity and spatial consistency. In the processing, first we extract connected image patches by triangulation among confidence matched point sets. Then our sampling method is executed in each connected image patch. We propose a judgment for matching points on the image patch boundary. Markov Random Field (MRF) model formulates the problem of dense matching as a Bayes decision task. Experiments are design to demonstrate the effective and efficiency of our method with active skin image data. [ABSTRACT FROM AUTHOR]

In the reconstruction of non-rigid three dimensional (3D) point cloud image features, the image data is large, which leads to the difficulty for analysis, and the achieving process is complex. In this paper, the reconstruction method for non-rigid 3D point cloud image features based on NURBS curve is proposed to collect non-rigid 3D point cloud image data, and achieve triangulation. The operation of point cloud data neighborhood search is limited in the local area, and the data of non-rigid 3D point cloud images are divided with the idea of space division. The least square plane is obtained by using preprocessed non-rigid 3D point cloud data, and the non-rigid 3D point cloud data is mapped and converted into a two dimensional (2D) point cloud model. The cumulative chord length parameterization method is adopted to introduce weighting factor for NURBS curve description using the rational polynomial function, based on the definition and properties of NURBS curve, point, line, surface reconstruction model is utilized to reconstruct the non-rigid 3D point cloud image feature. The simulation results show that the proposed method can reconstruct the 3D point cloud images feature with less time and the reconstruction effect is better than the Crust method. [ABSTRACT FROM AUTHOR]

This article proposes a real-time method that uses a single-view RGB-D input (a depth sensor integrated with a color camera) to simultaneously reconstruct a casual scene with a detailed geometry model, surface albedo, per-frame non-rigid motion, and per-frame low-frequency lighting, without requiring any template or motion priors. The key observation is that accurate scene motion can be used to integrate temporal information to recover the precise appearance, whereas the intrinsic appearance can help to establish true correspondence in the temporal domain to recover motion. Based on this observation, we first propose a shading-based scheme to leverage appearance information for motion estimation. Then, using the reconstructed motion, a volumetric albedo fusing scheme is proposed to complete and refine the intrinsic appearance of the scene by incorporating information from multiple frames. Since the two schemes are iteratively applied during recording, the reconstructed appearance and motion become increasingly more accurate. In addition to the reconstruction results, our experiments also show that additional applications can be achieved, such as relighting, albedo editing, and free-viewpoint rendering of a dynamic scene, since geometry, appearance, and motion are all reconstructed by our technique. [ABSTRACT FROM AUTHOR]

The effects of foundation rigidity and applied stress on the distributions of contact stresses beneath circular foundations resting on sandy soil were investigated in this paper. Three foundation cases were considered in this study; one case considered rigid foundations behavior with K value of 1 ( K is the relative soil/foundation stiffness index) and two cases considered non-rigid foundations behavior with K values of 0.05 and 0.005. The built-in material models of the Finite Element (FE) program ABAQUS were used to capture the stresses at the contact surfaces between the soil and the foundation at different applied loading levels (from 5%-100% of the ultimate bearing capacity of the foundation) for the three foundation cases. The FE analysis results indicated that the contact stresses ( q ) are strongly dependent on the relative soil/foundation stiffness index ( K ) and the ratio (q*) of the applied stress on the foundation (q) relative to the ultimate bearing capacity of the foundation (qu). For a rigid foundation, the contact stress distribution was found to have either a parabolic shape at the ultimate (q* = 1) with peak contact stress of 1.4q or a saddle shape for q* < 1 with peak contact stress that ranges from 1.3-1.5 q. For a non-rigid foundation, contact stresses distribution has a saddle shape at all q* values with a non-zero stresses at the edges and peak stress that ranges from 1.1-1.38 q. The numerical simulations results were then interpreted and presented in form of dimensionless charts, which in turn, will enable the estimation of the contact stress at any point within the contact area between the foundation and the soil. [ABSTRACT FROM AUTHOR]

Background: Coronary artery disease (CAD) is the leading cause of death worldwide. The registration of the coronary artery at different phases can help radiologists explore the motion patterns of the coronary artery and assist in the diagnosis of CAD. However, there is no automatic and easy-to-execute method to solve the missing data problem that occurs at the endpoints of the coronary artery tree. This paper proposed a non-rigid multi-constraint point set registration with redundant point removal (MPSR-RPR) algorithm to tackle this challenge., Methods: Firstly, the MPSR-RPR algorithm roughly registered two coronary artery point sets with the pre-set smoothness regularization parameter and Gaussian filter width value. The moving coherent, local feature, and the corresponding relationship between bifurcation point pairs were exploited as the constraints. Next, the spatial geometry information of the coronary artery was utilized to automatically recognize the vessel endpoints and to delete the redundant points of the coronary artery. Finally, the algorithm continued carrying out the multi-constraint registration with another group of the pre-set parameters to improve the alignment performance., Results: The experimental results demonstrated that the MPSR-RPR algorithm achieved a significantly lower mean value of the modified Hausdorff distance (MHD) compared to the other state-of-the-art methods for addressing the serious missing data in the left and right coronary arteries., Conclusion: This study demonstrated the effectiveness of the proposed algorithm in aligning coronary arteries, providing significant value in assisting in the diagnosis of coronary artery and myocardial lesions., Competing Interests: Declaration of competing interest No conflict of interest exists in the submission of this manuscript, and the manuscript is approved by all authors for publication in this journal. I would like to declare on behalf of my co-authors that the work described is original research that has not been published previously, and it is not under consideration for publication elsewhere, in whole or in part., (Copyright © 2023 Elsevier Ltd. All rights reserved.)

This article is a review of registration algorithms for use between ultrasound images (monomodal image-based ultrasound registration). Ultrasound is safe, inexpensive, and real-time, providing many advantages for clinical and scientific use on both humans and animals, but ultrasound images are also notoriously noisy and subject to several unique artifacts/distortions. This paper introduces the topic and unique aspects of ultrasound-to-ultrasound image registration, providing a broad introduction and summary of the literature and the field. Both theoretical and practical aspects are introduced. The first half of the paper is theoretical, organized according to the basic components of a registration framework, namely preprocessing, image-similarity metrics, optimizers, etc. It further subdivides these methods between those suitable for elastic (non-rigid) vs. inelastic (matrix) transforms. The second half of the paper is organized by anatomy and is practical in nature, presenting and discussing the complete published systems that have been validated for registration in specific anatomic regions. [ABSTRACT FROM AUTHOR]

Purpose:Mutual information (MI) has been a preferred choice of similarity measure for multi-modal image registration, but the accuracy and robustness of MI are not satisfactory as MI only considers the global intensity correlation while ignoring local and structural information. To address this problem, we combine MI with local and structural information. Method:We bring structural information extracted by a modified Accelerated Segment Test (FAST) algorithm into MI. Traditional FAST is transferred into 3 D for the first time, and the 3 D-FAST based structural information is added into MI as another channel, thereby incorporating spatial and geometric information with intensity information in the registration. Result:The robustness and accuracy of the proposed method were demonstrated in three experiments. The average registration errors of our method were 1.17, 1.33 and 1.20 compared to 1.47, 1.63 and 1.40 of LMI in T1-T2, T1-PD and T2-PD registration respectively. Discussion:In this paper, we use the structural similarity computed by 3 D-FAST as the conditional information to encode spatial and geometric cues into LMI. In all of these three experiments, our method shows to be more robust and accurate than common registration methods based on information theory. [ABSTRACT FROM PUBLISHER]

Head pose estimation under non-rigid face movement is particularly useful in applications relating to eye-gaze tracking in less constrained scenarios, where the user is allowed to move naturally during tracking. Existing vision-based head pose estimation methods often require accurate initialisation and tracking of specific facial landmarks, while methods that handle non-rigid face deformations typically necessitate a preliminary training phase prior to head pose estimation. In this paper, we propose a method to estimate the head pose in real-time from the trajectories of a set of feature points spread randomly over the face region, without requiring a training phase or model-fitting of specific facial features. Conversely, our method exploits the 3-dimensional shape of the surface of interest, recovered via shape and motion factorisation, in combination with Kalman and particle filtering to determine the contribution of each feature point to the estimation of head pose based on a variance measure. Quantitative and qualitative results reveal the capability of our method in handling non-rigid face movement without deterioration of the head pose estimation accuracy. [ABSTRACT FROM AUTHOR]

This paper proposes a framework based on harmonic mean normalized Laplace-Beltrami spectral descriptor. A series of experiments show that the harmonic mean normalization has better performance for non-rigid 3D retrieval, and it is robust to holes, local scaling, noise and sampling. To better distinguish shapes with fine or rough details, weighting method and fusion method are also employed. Weighting method reduces the negative impact of high-frequency information, and fusion method combines multi-level spectral information in both low and high frequencies. Our approach has better performance than other state-of-the-art methods on both retrieval accuracy and time consumption for stretched non-rigid 3D shapes. [ABSTRACT FROM AUTHOR]

A novel non-rigid object tracking based on interactive user-define marker and superpixel Gaussian kernel is proposed in this paper. In the initialization stage, instead of using the traditional bounding box to locate the targeted object, we have employed an interactive segmentation with user-defined marker to segment the object accurately in the first frame of the input video to avoid the background influence in the traditional bounding box. During the tracking stage, by using a Gaussian kernel as movement constraint, each superpixel is tracked independently to locate the object in the next frame. Experimental results show that the proposed method compared to state of the art methods can achieve better robustness and accuracy for various challenging video clips. [ABSTRACT FROM AUTHOR]

• We propose a novel method to learn isometry-invariant representation for point cloud which is strictly rotation-invariant and capable of tolerating pose variations. Both mathematical analyses and experimental results demonstrate that the proposed method can extract isometry-invariant representations for 3D shape analysis tasks without rotation augmentation. • In order to learn the local representation of point cloud, we build the local graph based the geodesic distance and propose the graph aggregation method to obtain the local feature, which is important for tolerating the shape variations in non-rigid shape analysis. • We construct a novel dataset, PKUnon-rigid, which is composed of non-rigid 3D objects. Based on it we can evaluate the capacity of several mainstream methods in terms of processing non-rigid shapes. 3D shape analysis has drawn broad attention due to its increasing demands in various fields. Despite that impressive performance has been achieved on several databases, most researchers focus their efforts on improving the performance of shape classification, retrieval, segmentation, etc. They neglect the fact that the disturbances, such as orientation and deformation, may impact much on the perception, restricting the capacity of generalizing to real applications where the prior of orientation and pose is often unknown. In this paper, we conduct shape analysis on point clouds and propose the point projection feature, which is rotation-invariant. Specifically, a novel architecture is designed to mine features of different levels. We adopt a PointNet-based backbone to extract global feature for the point cloud, and the graph aggregation operation to perceive local pose variance in the Euclidean space or geodesic space. An efficient key point descriptor is designed to assign each point with different response and help recognize the overall geometry. Furthermore, a novel dataset, PKUnon-rigid, is built that is composed of non-rigid 3D objects, based on which we evaluate the capacity of several mainstream methods in terms of processing non-rigid shapes. Mathematical analyses and experimental results demonstrate that the proposed method can extract isometry-invariant representations for 3D shape analysis tasks without rotation augmentation, and outperforms other state-of-the-art methods. The proposed dataset is publicly available at https://github.com/tasx0823/PKUnon-rigid. [ABSTRACT FROM AUTHOR]

In previous work on point registration, the input point sets are often represented using Gaussian mixture models and the registration is then addressed through a probabilistic approach, which aims to exploit global relationships on the point sets. For non-rigid shapes, however, the local structures among neighboring points are also strong and stable and thus helpful in recovering the point correspondence. In this paper, we formulate point registration as the estimation of a mixture of densities, where local features, such as shape context, are used to assign the membership probabilities of the mixture model. This enables us to preserve both global and local structures during matching. The transformation between the two point sets is specified in a reproducing kernel Hilbert space and a sparse approximation is adopted to achieve a fast implementation. Extensive experiments on both synthesized and real data show the robustness of our approach under various types of distortions, such as deformation, noise, outliers, rotation, and occlusion. It greatly outperforms the state-of-the-art methods, especially when the data is badly degraded. [ABSTRACT FROM PUBLISHER]

The objective of the work is to model the shape of the sinusoidal shape of regular water waves generated in a laboratory flume. The waves are traveling in time and render a smooth surface, with no white caps or foam. Two methods are proposed, treating the water as a diffuse and specular surface, respectively. In either case, the water is presumed to take the shape of a traveling sine wave, reducing the task of the 3D reconstruction to resolve the wave parameters. The first conceived method performs the modeling part purely in 3D space. Having triangulated the points in a separate phase via bundle adjustment, a sine wave is fitted into the data in a least squares manner. The second method presents a more complete approach for the entire calculation workflow beginning in the image space. The water is perceived as a specular surface, and the traveling specularities are the only observations visible to the cameras, observations that are notably single image. The depth ambiguity is removed given additional constraints encoded within the law of reflection and the modeled parametric surface. The observation and constraint equations compose a single system of equations that is solved with the method of least squares adjustment. The devised approaches are validated against the data coming from a capacitive level sensor and on physical targets floating on the surface. The outcomes agree to a high degree. [ABSTRACT FROM AUTHOR]