Search

Image-based patient-specific modelling of hemodynamics are gaining increased popularity as a diagnosis and outcome prediction solution for a variety of cardiovascular diseases. While their potential to improve diagnostic capabilities and thereby clinical outcome is widely recognized, these methods require considerable computational resources since they are mostly based on conventional numerical methods such as computational fluid dynamics (CFD). As an alternative to the numerical methods, we propose a machine learning (ML) based approach to calculate patient-specific hemodynamic parameters. Compared to CFD based methods, our approach holds the benefit of being able to calculate a patient-specific hemodynamic outcome instantly with little need for computational power. In this proof-of-concept study, we present a deep artificial neural network (ANN) capable of computing hemodynamics for patients with aortic coarctation in a centerline aggregated (i.e., locally averaged) form. Considering the complex relation between vessels shape and hemodynamics on the one hand and the limited availability of suitable clinical data on the other, a sufficient accuracy of the ANN may however not be achieved with available data only. Another key aspect of this study is therefore the successful augmentation of available clinical data. Using a statistical shape model, additional training data was generated which substantially increased the ANN's accuracy, showcasing the ability of ML based methods to perform in-silico modelling tasks previously requiring resource intensive CFD simulations.

International audience; Computer models of the heart are of increasing interest for clinical applications due to their discriminative and predictive abilities. However the number of simulation parameters in these models can be high and expert knowledge is required to properly design studies involving these models, and analyse the results. In particular it is important to know how the parameters vary in various clinical or physiological settings. In this paper we build a data-driven model of cardiovascular parameter evolution during digestion, from a clinical study involving more than 80 patients. We first present a method for longitudinal parameter estimation in 3D cardiac models, which we apply to 21 patient-specific hearts geometries at two instants of the study, for 6 parameters (two fixed and four time-varying parameters). From these personalised hearts, we then extract and validate a law which links the changes of cardiac output and heart rate under constant arterial pressure to the evolution of these parameters, thus enabling the fast simulation of hearts during digestion for future patients.

International audience; Personalised 3D modelling of the heart is of increasing interest in order to better characterise pathologies and predict evolution. The personalisation consists in estimating the parameter values of an electromechanical model in order to reproduce the observed cardiac motion. However, the number of parameters in these models can be high and their estimation may not be unique. This variability can be an obstacle to further analyse the estimated parameters and for their clinical interpretation. In this paper we present a method to perform consistent estimations of electromechanical parameters with prior probabilities on the estimated values, which we apply on a large database of 84 different heartbeats. We show that the use of priors reduces considerably the variance in the estimated parameters, enabling better conditioning of the parameters for further analysis of the cardiac function. This is demonstrated by the application to longitudinal data of paediatric cardiomyopathies, where the estimated parameters provide additional information on the pathology and its evolution.

International audience; In this paper, we propose an innovative approach for registration based on the deterministic prediction of the parameters from both images instead of the optimization of a energy criteria. The method relies on a fully convolutional network whose architecture consists of contracting layers to detect relevant features and a symmetric expanding path that matches them together and outputs the transformation parametriza-tion. Whereas convolutional networks have seen a widespread expansion and have been already applied to many medical imaging problems such as segmentation and classification, its application to registration has so far faced the challenge of defining ground truth data on which to train the algorithm. Here, we present a novel training strategy to build reference deformations which relies on the registration of segmented regions of interest. We apply this methodology to the problem of inter-patient heart registration and show an important improvement over a state of the art optimization based algorithm. Not only our method is more accurate but it is also faster-registration of two 3D-images taking less than 30ms second on a GPU-and more robust to outliers.

International audience; Understanding and modeling liver biomechanics represents a significant challenge due to its complex nature. In this paper, we tackle this issue in the context of real time surgery simulation where a compromise between biomechanical accuracy and computational efficiency must be found. We describe a realistic liver model including hyperelasticity, porosity and viscosity that is implemented within an implicit time integration scheme. To optimize its computation, we introduce the Multiplicative Jacobian Energy Decomposition (MJED) method for discretizing hyperelastic materials on linear tetrahedral meshes which leads to faster matrix assembly than the standard Finite Element Method. Viscohyperelasticity is modeled by Prony series while the mechanical eff ect of liver perfusion is represented with a linear Darcy law. Dynamic mechanical analysis has been performed on 60 porcine liver samples in order to identify some visco-elastic parameters. Finally, we show that liver deformation can be simulated in real-time on a coarse mesh and study the relative eff ects of the hyperelastic, viscous and porous components on the liver biomechanics.

The fusion of image data from trans-esophageal echography (TEE) and X-ray fluoroscopy is attracting increasing interest in minimally-invasive treatment of structural heart disease. In order to calculate the needed transformation between both imaging systems, we employ a discriminative learning (DL) based approach to localize the TEE transducer in X-ray images. The successful application of DL methods is strongly dependent on the available training data, which entails three challenges: (1) the transducer can move with six degrees of freedom meaning it requires a large number of images to represent its appearance, (2) manual labeling is time consuming, and (3) manual labeling has inherent errors. This paper proposes to generate the required training data automatically from a single volumetric image of the transducer. In order to adapt this system to real X-ray data, we use unlabeled fluoroscopy images to estimate differences in feature space density and correct covariate shift by instance weighting. Two approaches for instance weighting, probabilistic classification and Kullback–Leibler importance estimation (KLIEP), are evaluated for different stages of the proposed DL pipeline. An analysis on more than 1900 images reveals that our approach reduces detection failures from 7.3% in cross validation on the test set to zero and improves the localization error from 1.5 to 0.8 mm. Due to the automatic generation of training data, the proposed system is highly flexible and can be adapted to any medical device with minimal efforts.

2D/3D registration is a well known technique in medical imaging for combining pre-operative volume data with live fluoroscopy. A common issue of this type of algorithms is that out-of-plane parameters are hard to determine. One solution to overcome this issue is the use of X-ray images from two angulations. However, performing in-plane transformation in one image destroys the registration in the other image, particularly if the angulations are smaller than 90 degrees apart. Our main contribution is the automation of a novel registration approach. It handles translation and rotation of a volume in a way that in-plane parameters are kept invariant and independent of the angle offset between both projections in a double-oblique setting. Our approach yields more robust and partially faster registration results, compared to conventional methods, especially in case of object movement. It was successfully tested on clinical data for fusion of transesophageal ultrasound and X-ray.

Ultrasound and X-ray are two facilitating imaging modalities in the field of transcatheter-based minimally invasive procedures in structural heart disease. X-ray fluoroscopy provides excellent instrument imaging and ultrasound shows high-quality images of soft tissue. A fusion of both modalities can potentially improve the surgical workflow and the catheter navigation. A current approach shows the fusion of X-ray fluoroscopy with trans-esophageal echo (TEE) with the help of 2D/3D registration. An ultrasound probe model is registered to X-ray images which inherently provides a registration of ultrasound images to X-ray. In this paper, we evaluate the accuracy and the performance of four optimizing algorithms (Powell, Nelder-Mead, BFGS, CMA-ES) while registering digitally reconstructed radiographs (DRR) of the model to X-ray images. The DRRs were generated by mesh-rendering, not by ray casting. The optimizers show significant differences in accuracy and runtime.

One of the main challenges in computer-assisted soft tissue surgery is the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilities by observing beyond exposed tissue surfaces. A new approach to marker-less guidance involves capturing the intra-operative patient anatomy with a range image device and doing a shape-based registration. However, as the target organ is only partially visible, typically does not provide salient features and underlies severe non-rigid deformations, surface matching in this context is extremely challenging. Furthermore, the intra-operatively acquired surface data may be subject to severe systematic errors and noise. To address these issues, we propose a new approach to establishing surface correspondences, which can be used to initialize fine surface matching algorithms in the context of intra-operative shape-based registration. Our method does not require any prior knowledge on the relative poses of the input surfaces to each other, does not rely on the detection of prominent surface features, is robust to noise and can be used for overlapping surfaces. It takes into account (1) similarity of feature descriptors, (2) compatibility of multiple correspondence pairs, as well as (3) the spatial configuration of the entire correspondence set. We evaluate the algorithm on time-of-flight (ToF) data from porcine livers in a respiratory liver motion simulator. In all our experiments the alignment computed from the established surface correspondences yields a registration error below 1 cm and is thus well suited for initializing fine surface matching algorithms for intra-operative soft-tissue registration.

We propose a top-down fully automatic 3D vertebra segmentation algorithm using global shape-related as well as local appearance-related prior information. The former is brought into the system by a global statistical shape model built from annotated training data, i.e., annotated CT volumes. The latter is handled by a machine learning-based component, i.e., a boundary detector, providing a strong discriminative model for vertebra surface appearance by making use of local context-encoding features. This boundary detector, which is essentially a probabilistic boosting-tree classifier, is also learnt from annotated training data. Contextual information is taken into account by representing vertebra surface candidate voxels with high-dimensional vectors of 3D steerable features derived from the observed volume intensities. Our system does not only consider the body of the individual vertebrae but also the spinal processes. Before segmentation, the image parts depicting individual vertebrae are spatially normalized with respect to their bounding box information in terms of translation, orientation, and scale leading to more accurate results. We evaluate segmentation accuracy on 7 CT volumes each depicting 22 vertebrae. The results indicate a symmetric point-to-mesh surface error of 1.37 ± 0.37 mm, which matches the current state-of-the-art.

The fusion of image data from trans-esophageal echography (TEE) and X-ray fluoroscopy is attracting increasing interest in minimally-invasive treatment of structural heart disease. In order to calculate the needed transform between both imaging systems, we employ a discriminative learning based approach to localize the TEE transducer in X-ray images. Instead of time-consuming manual labeling, we generate the required training data automatically from a single volumetric image of the transducer. In order to adapt this system to real X-ray data, we use unlabeled fluoroscopy images to estimate differences in feature space density and correct covariate shift by instance weighting. An evaluation on more than 1900 images reveals that our approach reduces detection failures by 95% compared to cross validation on the test set and improves the localization error from 1.5 to 0.8 mm. Due to the automatic generation of training data, the proposed system is highly flexible and can be adapted to any medical device with minimal efforts.

Mitral regurgitation is a wide spread problem. For successful surgical treatment quantification of the mitral annulus, especially its diameter, is essential. Time resolved 3D transesophageal echocardiography (TEE) is suitable for this task. Yet, manual measurement in four dimensions is extremely time consuming, which confirms the need for automatic quantification methods. The method we propose is capable of automatically detecting the cardiac cycle (systole or diastole) for each time step and measuring the mitral annulus diameter. This is done using total variation noise filtering, the graph cut segmentation algorithm and morphological operators. An evaluation took place using expert measurements on 4D TEE data of 13 patients. The cardiac cycle was detected correctly on 78% of all images and the mitral annulus diameter was measured with an average error of 3.08 mm. Its full automatic processing makes the method easy to use in the clinical workflow and it provides the surgeon with helpful information.

Range imaging modalities, such as time-of-flight cameras (ToF), are becoming very popular for the acquisition of intra-operative data, which can be used for registering the patient's anatomy with pre-operative data, such as 3D images generated by computed tomographies (CT) or magnetic resonance imaging (MRI). However, due to the distortions that appear because of the different acquisition principles of the input surfaces, the noise, and the deformations that may occur in the intra-operative environment, we face different surface properties for points lying on the same anatomical locations and unreliable feature points detection, which are crucial for most surface matching algorithms. In order to overcome these issues, we present a method for automatically finding correspondences between surfaces that searches for minimally deformed configurations. For this purpose, an error metric that expresses the reliability of a correspondence set based on its spatial configuration is employed. The registration error is minimized by a combinatorial analysis through search-trees. Our method was evaluated with real and simulated ToF and CT data, and showed to be reliable for the registration of partial multi-modal surfaces with noise and distortions.

The segmentation of tree-like tubular structures such as coronary arteries and airways is an essential step for many 3D medical imaging applications. Statistical tracking techniques for the extraction of elongated structures have received considerable attention in recent years due to their robustness against image noise and pathological changes. However, most tracking methods are limited to a specific application and do not support branching structures efficiently. In this work, we present a novel statistical tracking approach for the extraction of different types of tubular structures with ringlike cross-sections. Domain-specific knowledge is learned from training data sets and integrated into the tracking process by simple adaption of parameters. In addition, an efficient branching detection algorithm is presented. This approach was evaluated by extracting coronary arteries from 32 CTA data sets and distal airways from 20 CT scans. These data sets were provided by the organizers of the workshop '3D Segmentation in the Clinic: A Grand Challenge II-Coronary Artery Tracking (CAT08)' and 'Extraction of Airways from CT 2009 (EXACT' 09)'. On average, 81.5% overlap and 0.51 mm accuracy for the tracking of coronary arteries were achieved. For the extraction of airway trees, 51.3% of the total tree length, 53.6% of the total number of branches and a 4.98% false positive rate were attained. In both experiments, our approach is comparable to state-of-the-art methods.

Time-of-flight (ToF) cameras are a novel, fast, and robust means for intra-operative 3D surface acquisition. They acquire surface information (range images) in real-time. In the intra-operative registration context, these surfaces must be matched to pre-operative CT or MR surfaces, using so called descriptors, which represent surface characteristics. We present a framework for local and global multi-modal comparison of surface descriptors and characterize the differences between ToF and CT data in an in vitro experiment. The framework takes into account various aspects related to the surface characteristics and does not require high resolution input data in order to establish appropriate correspondences. We show that the presentation of local and global comparison data allows for an accurate assessment of ToF-CT discrepancies. The information gained from our study may be used for developing ToF pre-processing and matching algorithms, or for improving calibration procedures for compensating systematic distance errors. The framework is available in the open-source platform Medical Imaging Interaction Toolkit (MITK).

Obesity is an increasing problem in the western world and triggers diseases like cancer, type two diabetes, and cardiovascular diseases. In recent years, magnetic resonance imaging (MRI) has become a clinically viable method to measure the amount and distribution of adipose tissue (AT) in the body. However, analysis of MRI images by manual segmentation is a tedious and time-consuming process. In this paper, we propose a semi-automatic method to quantify the amount of different AT types from whole-body MRI data with less user interaction. Initially, body fat is extracted by automatic thresholding. A statistical shape model of the abdomen is then used to differentiate between subcutaneous and visceral AT. Finally, fat in the bone marrow is removed using morphological operators. The proposed method was evaluated on 15 whole-body MRI images using manual segmentation as ground truth for adipose tissue. The resulting overlap for total AT was 93.7% ± 5.5 with a volumetric difference of 7.3% ± 6.4. Furthermore, we tested the robustness of the segmentation results with regard to the initial, interactively defined position of the shape model. In conclusion, the developed method proved suitable for the analysis of AT distribution from whole-body MRI data. For large studies, a fully automatic version of the segmentation procedure is expected in the near future.

Vessel tree tracking is an important and challenging task for many medical applications. This paper presents a novel bifurcation detection algorithm for Bayesian tracking of vessel trees. Based on a cylindrical model, we introduce a bifurcation metric that yields minimal values at potential branching points. This approach avoids searching for bifurcations in every iteration of the tracking process (as proposed by prior works) and is therefore computationally more efficient. We use the same geometric model for the bifurcation metric as for the tracking; no specific bifurcation model is needed. In a preliminary evaluation of our method on 8 CTA datasets of coronary arteries, all side branches and 95.8% of the main branches were detected correctly.

Due to noise and artifacts often encountered in medical images, segmenting objects in these is one of the most challenging tasks in medical image analysis. Model-based approaches like statistical shape models (SSMs) incorporate prior knowledge that supports object detection in case of in-complete evidence from image data. In this paper, we present a method to increase information of the object's shape in problematic image areas by incorporating mutual shape information from other entities in the image. This is done by using a common shape space of multiple objects as additional restriction. Two different approaches to implement mutual shape information are presented. Evaluation was performed on nine cardiac images by simultaneous segmentation of the epi- and endocardium of the left heart ventricle using the proposed methods. The results show that the segmentation quality is improved with both methods. For the better one, the average surface distance error is approx. 40% lower.

International audience; Simulating soft tissues in real time is a signi cant challenge since a compromise between biomechanical accuracy and computational efficiency must be found. In this paper, we propose a new discretization method, the Multiplicative Jacobian Energy Decomposition (MJED) which is an alternative to the classical Galerkin FEM (Finite Element Method) formulation. This method for discretizing non-linear hyperelastic materials on linear tetrahedral meshes leads to faster sti ffness matrix assembly for a large variety of isotropic and anisotropic materials. We show that our new approach, implemented within an implicit time integration scheme, can lead to fast and realistic liver deformations including hyperelasticity, porosity and viscosity.

International audience; This chapter starts with a brief introduction into model-based segmentation, explaining the basic concepts and different approaches. Subsequently, two segmentation approaches are presented in more detail: First, the method of deformable simplex meshes is described, explaining the special properties of the simplex mesh and the formulation of the internal forces. Common choices for image forces are presented, and how to evolve the mesh to adapt to certain structures. Second, the method of point-based statistical shape models (SSMs) is described. The model construction process is explained and the point correspondence problem is treated in detail. Several approaches of how gray level appearance can be modeled are presented, and search algorithms that use this knowledge to segment the modeled structures in new images are described.

The random walker algorithm is a graph-based segmentation method that has become popular over the past fewyears. The basis of the algorithm is a large, sparsely occupied system of linear equations, whose size correspondsto the number of voxels in the image. To solve these systems, typically comprised of millions of equations,the computational performance of conventional numerical solution methods (e.g. Gauss-Seidel) is no longersatisfactory. An alternative method that has been described previously for solving 2D random walker problemsis the geometrical multigrid method. In this paper, we present a geometrical multigrid approach for the 3Drandom walker problem. Our approach features an optimized calculation of the required Galerkin product anda robust smoothing using the ILU method. To reach better convergence rates, the multigrid solver is used as apreconditioner for the conjugate gradient solver. We co mpared the performance of our new multigrid approachwith the conjugate gradient solver on “ve MRI lung images with a resolution of 96 × 128 × 52 voxels. Initialresults show an increasing in speed of up to four times, reducing the average computation time from six minutesto less than two minutes when using our proposed approach. Employing a multigrid solver for the random walkeralgorithm thus permits accurate interactive segmentation with fewer delays.Keywords: segmentation, random walker, multigrid

This paper presents a comparison study between 10 automatic and six interactive methods for liver segmentation from contrast-enhanced CT images. It is based on results from the "MICCAI 2007 Grand Challenge" workshop, where 16 teams evaluated their algorithms on a common database. A collection of 20 clinical images with reference segmentations was provided to train and tune algorithms in advance. Participants were also allowed to use additional proprietary training data for that purpose. All teams then had to apply their methods to 10 test datasets and submit the obtained results. Employed algorithms include statistical shape models, atlas registration, level-sets, graph-cuts and rule-based systems. All results were compared to reference segmentations five error measures that highlight different aspects of segmentation accuracy. All measures were combined according to a specific scoring system relating the obtained values to human expert variability. In general, interactive methods reached higher average scores than automatic approaches and featured a better consistency of segmentation quality. However, the best automatic methods (mainly based on statistical shape models with some additional free deformation) could compete well on the majority of test images. The study provides an insight in performance of different segmentation approaches under real-world conditions and highlights achievements and limitations of current image analysis techniques.

International audience; We present an efficient finite element method to simulate a transversely isotropic non-linear material for ligaments. The approach relies on tetrahedral elements and exploits the geometry to optimize computation of the derivatives of the strain energy. To better support incompressibilty, deviatoric and dilational responses are uncoupled and a penalty term controls volume preservation. We derive stress and elasticity tensors required for implicit solvers and verify our model against the FEBio software using a variety of load scenarios with synthetic shapes. The maximum node positioning error for ligament materials is < 5% for strains under physiological conditions. To generate subject-specific ligament models, we propose a novel technique to estimate fiber orientation from segmented ligament geometry. The approach is based on an automatic centerline extraction and generation of the corresponding diffusion field. We present results for a medial collateral ligament segmented from standard MRI data. Results show the general viability of the method, but also the limitations of current MRI acquisitions. In the future, we hope to employ the presented techniques for real-time simulation of knee surgery.

With many tumor entities, quantitative assessment of lymph node growth over time is important to make therapy choices or to evaluate new therapies. The clinical standard is to document diameters on transversal slices, which is not the best measure for a volume. We present a new algorithm to segment (metastatic) lymph nodes and evaluate the algorithm with 29 lymph nodes in clinical CT images. The algorithm is based on a deformable surface search, which uses statistical shape models to restrict free deformation. To model lymph nodes, we construct an ellipsoid shape model, which strives for a surface with strong gradients and user-defined gray values. The algorithm is integrated into an application, which also allows interactive correction of the segmentation results. The evaluation shows that the algorithm gives good results in the majority of cases and is comparable to time-consuming manual segmentation. The median volume error was 10.1% of the reference volume before and 6.1% after manual correction. Integrated into an application, it is possible to perform lymph node volumetry for a whole patient within the 10 to 15 minutes time limit imposed by clinical routine.

Statistical shape models have become a fast and robust method for segmentation of anatomical structures in medical image volumes. In clinical practice, however, pathological cases and image artifacts can lead to local deviations of the detected contour from the true object boundary. These deviations have to be corrected manually. We present an intuitively applicable solution for surface interaction based on Gaussian deformation kernels. The method is evaluated by two radiological experts on segmentations of the liver in contrast-enhanced CT images and of the left heart ventricle (LV) in MRI data. For both applications, five datasets are segmented automatically using deformable shape models, and the resulting surfaces are corrected manually. The interactive correction step improves the average surface distance against ground truth from 2.43mm to 2.17mm for the liver, and from 2.71mm to 1.34mm for the LV. We expect this method to raise the acceptance of automatic segmentation methods in clinical application.

This paper presents an evaluation of the performance of a three-dimensional Active Shape Model (ASM) to segment the liver in 48 clinical CT scans. The employed shape model is built from 32 samples using an optimization approach based on the minimum description length (MDL). Three different gray-value appearance models (plain intensity, gradient and normalized gradient profiles) are created to guide the search. The employed segmentation techniques are ASM search with 10 and 30 modes of variation and a deformable model coupled to a shape model with 10 modes of variation. To assess the segmentation performance, the obtained results are compared to manual segmentations with four different measures (overlap, average distance, RMS distance and ratio of deviations larger 5mm). The only appearance model delivering usable results is the normalized gradient profile. The deformable model search achieves the best results, followed by the ASM search with 30 modes. Overall, statistical shape modeling delivers very promising results for a fully automated segmentation of the liver.

Active Shape Models are a popular method for segmenting three-dimensional medical images. To obtain the required landmark correspondences, various automatic approaches have been proposed. In this work, we present an improved version of minimizing the description length (MDL) of the model. To initialize the algorithm, we describe a method to distribute landmarks on the training shapes using a conformal parameterization function. Next, we introduce a novel procedure to modify landmark positions locally without disturbing established correspondences. We employ a gradient descent optimization to minimize the MDL cost function, speeding up automatic model building by several orders of magnitude when compared to the original MDL approach. The necessary gradient information is estimated from a singular value decomposition, a more accurate technique to calculate the PCA than the commonly used eigendecomposition of the covariance matrix. Finally, we present results for several synthetic and real-world datasets demonstrating that our procedure generates models of significantly better quality in a fraction of the time needed by previous approaches.

Computer-assisted surgery aims at a decreased surgical risk and a reduced recovery time of patients. However, its use is still limited to complex cases because of the high effort. It is often caused by the extensive medical image analysis. Especially, image segmentation requires a lot of manual work. Surgeons and radiologists are suffering from usability problems of many workstations. In this work, we present a dedicated workplace for interactive segmentation integratd within the CHILI (tele-)radiology system. The software comes with a lot of improvements with respect to its graphical user interface, the segmentation process and the segmentatin methods. We point out important software requirements and give insight into the concepts which were implemented. Further examples and applications illustrate the software system.