Search

Lung ultrasound (LUS) is an important imaging modality used by emergency physicians to assess pulmonary congestion at the patient bedside. B-line artifacts in LUS videos are key findings associated with pulmonary congestion. Not only can the interpretation of LUS be challenging for novice operators, but visual quantification of B-lines remains subject to observer variability. In this work, we investigate the strengths and weaknesses of multiple deep learning approaches for automated B-line detection and localization in LUS videos. We curate and publish, BEDLUS, a new ultrasound dataset comprising 1,419 videos from 113 patients with a total of 15,755 expert-annotated B-lines. Based on this dataset, we present a benchmark of established deep learning methods applied to the task of B-line detection. To pave the way for interpretable quantification of B-lines, we propose a novel "single-point" approach to B-line localization using only the point of origin. Our results show that (a) the area under the receiver operating characteristic curve ranges from 0.864 to 0.955 for the benchmarked detection methods, (b) within this range, the best performance is achieved by models that leverage multiple successive frames as input, and (c) the proposed single-point approach for B-line localization reaches an F 1 -score of 0.65, performing on par with the inter-observer agreement. The dataset and developed methods can facilitate further biomedical research on automated interpretation of lung ultrasound with the potential to expand the clinical utility.

Deep learning-based analysis of high-frequency, high-resolution micro-ultrasound data shows great promise for prostate cancer (PCa) detection. Previous approaches to analysis of ultrasound data largely follow a supervised learning (SL) paradigm. Ground truth labels for ultrasound images used for training deep networks often include coarse annotations generated from the histopathological analysis of tissue samples obtained via biopsy. This creates inherent limitations on the availability and quality of labeled data, posing major challenges to the success of SL methods. However, unlabeled prostate ultrasound data are more abundant. In this work, we successfully apply self-supervised representation learning to micro-ultrasound data. Using ultrasound data from 1028 biopsy cores of 391 subjects obtained in two clinical centers, we demonstrate that feature representations learned with this method can be used to classify cancer from noncancer tissue, obtaining an AUROC score of 91% on an independent test set. To the best of our knowledge, this is the first successful end-to-end self-SL (SSL) approach for PCa detection using ultrasound data. Our method outperforms baseline SL approaches, generalizes well between different data centers, and scales well in performance as more unlabeled data are added, making it a promising approach for future research using large volumes of unlabeled data. Our code is publicly available at https://www.github.com/MahdiGilany/SSL_micro_ultrasound.

This paper presents U-LanD, a framework for automatic detection of landmarks on key frames of the video by leveraging the uncertainty of landmark prediction. We tackle a specifically challenging problem, where training labels are noisy and highly sparse. U-LanD builds upon a pivotal observation: a deep Bayesian landmark detector solely trained on key video frames, has significantly lower predictive uncertainty on those frames vs. other frames in videos. We use this observation as an unsupervised signal to automatically recognize key frames on which we detect landmarks. As a test-bed for our framework, we use ultrasound imaging videos of the heart, where sparse and noisy clinical labels are only available for a single frame in each video. Using data from 4,493 patients, we demonstrate that U-LanD can exceedingly outperform the state-of-the-art non-Bayesian counterpart by a noticeable absolute margin of 42% in R 2 score, with almost no overhead imposed on the model size.

In echocardiography (echo), an electrocardiogram (ECG) is conventionally used to temporally align different cardiac views for assessing critical measurements. However, in emergencies or point-of-care situations, acquiring an ECG is often not an option, hence motivating the need for alternative temporal synchronization methods. Here, we propose Echo-SyncNet, a self-supervised learning framework to synchronize various cross-sectional 2D echo series without any human supervision or external inputs. The proposed framework takes advantage of two types of supervisory signals derived from the input data: spatiotemporal patterns found between the frames of a single cine (intra-view self-supervision) and interdependencies between multiple cines (inter-view self-supervision). The combined supervisory signals are used to learn a feature-rich and low dimensional embedding space where multiple echo cines can be temporally synchronized. Two intra-view self-supervisions are used, the first is based on the information encoded by the temporal ordering of a cine (temporal intra-view) and the second on the spatial similarities between nearby frames (spatial intra-view). The inter-view self-supervision is used to promote the learning of similar embeddings for frames captured from the same cardiac phase in different echo views. We evaluate the framework with multiple experiments: 1) Using data from 998 patients, Echo-SyncNet shows promising results for synchronizing Apical 2 chamber and Apical 4 chamber cardiac views, which are acquired spatially perpendicular to each other; 2) Using data from 3070 patients, our experiments reveal that the learned representations of Echo-SyncNet outperform a supervised deep learning method that is optimized for automatic detection of fine-grained cardiac cycle phase; 3) We go one step further and show the usefulness of the learned representations in a one-shot learning scenario of cardiac key-frame detection. Without any fine-tuning, key frames in 1188 validation patient studies are identified by synchronizing them with only one labeled reference cine. We do not make any prior assumption about what specific cardiac views are used for training, and hence we show that Echo-SyncNet can accurately generalize to views not present in its training set. Project repository: github.com/fatemehtd/Echo-SyncNet>.

Fully convolutional neural networks (FCNs), and in particular U-Nets, have achieved state-of-the-art results in semantic segmentation for numerous medical imaging applications. Moreover, batch normalization and Dice loss have been used successfully to stabilize and accelerate training. However, these networks are poorly calibrated i.e. they tend to produce overconfident predictions for both correct and erroneous classifications, making them unreliable and hard to interpret. In this paper, we study predictive uncertainty estimation in FCNs for medical image segmentation. We make the following contributions: 1) We systematically compare cross-entropy loss with Dice loss in terms of segmentation quality and uncertainty estimation of FCNs; 2) We propose model ensembling for confidence calibration of the FCNs trained with batch normalization and Dice loss; 3) We assess the ability of calibrated FCNs to predict segmentation quality of structures and detect out-of-distribution test examples. We conduct extensive experiments across three medical image segmentation applications of the brain, the heart, and the prostate to evaluate our contributions. The results of this study offer considerable insight into the predictive uncertainty estimation and out-of-distribution detection in medical image segmentation and provide practical recipes for confidence calibration. Moreover, we consistently demonstrate that model ensembling improves confidence calibration.

Uncertainty of labels in clinical data resulting from intra-observer variability can have direct impact on the reliability of assessments made by deep neural networks. In this paper, we propose a method for modelling such uncertainty in the context of 2D echocardiography (echo), which is a routine procedure for detecting cardiovascular disease at point-of-care. Echo imaging quality and acquisition time is highly dependent on the operator's experience level. Recent developments have shown the possibility of automating echo image quality quantification by mapping an expert's assessment of quality to the echo image via deep learning techniques. Nevertheless, the observer variability in the expert's assessment can impact the quality quantification accuracy. Here, we aim to model the intra-observer variability in echo quality assessment as an aleatoric uncertainty modelling regression problem with the introduction of a novel method that handles the regression problem with categorical labels. A key feature of our design is that only a single forward pass is sufficient to estimate the level of uncertainty for the network output. Compared to the 0.11 ± 0.09 absolute error (in a scale from 0 to 1) archived by the conventional regression method, the proposed method brings the error down to 0.09 ± 0.08, where the improvement is statistically significant and equivalents to 5.7% test accuracy improvement. The simplicity of the proposed approach means that it could be generalized to other applications of deep learning in medical imaging, where there is often uncertainty in clinical labels.

Current deep supervised learning methods typically require large amounts of labeled data for training. Since there is a significant cost associated with clinical data acquisition and labeling, medical datasets used for training these models are relatively small in size. In this paper, we aim to alleviate this limitation by proposing a variational generative model along with an effective data augmentation approach that utilizes the generative model to synthesize data. In our approach, the model learns the probability distribution of image data conditioned on a latent variable and the corresponding labels. The trained model can then be used to synthesize new images for data augmentation. We demonstrate the effectiveness of the approach on two independent clinical datasets consisting of ultrasound images of the spine and magnetic resonance images of the brain. For the spine dataset, a baseline and a residual model achieve an accuracy of 85% and 92%, respectively, using our method compared to 78% and 83% using a conventional training approach for image classification task. For the brain dataset, a baseline and a U-net network achieve an accuracy of 84% and 88%, respectively, in Dice coefficient in tumor segmentation compared to 80% and 83% for the convention training approach.

Clear visualization of spine structures in ultrasound imaging is difficult due to factors such as specular reflection, off-axis energy, and reverberation artifacts. The received channel data from the spine are often tilted even after delay correction, resulting in signal cancellation during the beamforming process. Conventional beamformers are not designed to tackle this issue. This paper proposes a closed-loop beamforming method which feeds back the location of the spine to the beamforming process so that backscattered bone signals can be aligned prior to the beamforming. To suppress the weak soft tissue and reverberation artifacts and increase the contrast of bones, a tensor-based filtering is employed prior to the cross-correlation-based alignment. Directional filtering is also employed to improve the bone surface detection. Phantom studies show improvement on the sharpness of the spine without shape distortion. In vivo results confirm significant contrast improvement of spinal structures. Compared with the conventional delay-and-sum beamforming, the proposed method improves the contrast ratio (CR) of the spine from 0.56 to 0.96. The 6-dB width of bone surfaces is also reduced by 51%.

Accurate detection of end-systolic (ES) and end-diastolic (ED) frames in an echocardiographic cine series can be difficult but necessary pre-processing step for the development of automatic systems to measure cardiac parameters. The detection task is challenging due to variations in cardiac anatomy and heart rate often associated with pathological conditions. We formulate this problem as a regression problem and propose several deep learning-based architectures that minimize a novel global extrema structured loss function to localize the ED and ES frames. The proposed architectures integrate convolution neural networks (CNNs)-based image feature extraction model and recurrent neural networks (RNNs) to model temporal dependencies between each frame in a sequence. We explore two CNN architectures: DenseNet and ResNet, and four RNN architectures: long short-term memory, bi-directional LSTM, gated recurrent unit (GRU), and Bi-GRU, and compare the performance of these models. The optimal deep learning model consists of a DenseNet and GRU trained with the proposed loss function. On average, we achieved 0.20 and 1.43 frame mismatch for the ED and ES frames, respectively, which are within reported inter-observer variability for the manual detection of these frames.

Image guidance improves tissue sampling during biopsy by allowing the physician to visualize the tip and trajectory of the biopsy needle relative to the target in MRI, CT, ultrasound, or other relevant imagery. This paper reports a system for fast automatic needle tip and trajectory localization and visualization in MRI that has been developed and tested in the context of an active clinical research program in prostate biopsy. To the best of our knowledge, this is the first reported system for this clinical application and also the first reported system that leverages deep neural networks for segmentation and localization of needles in MRI across biomedical applications. Needle tip and trajectory were annotated on 583 T2-weighted intra-procedural MRI scans acquired after needle insertion for 71 patients who underwent transperineal MRI-targeted biopsy procedure at our institution. The images were divided into two independent training-validation and test sets at the patient level. A deep 3-D fully convolutional neural network model was developed, trained, and deployed on these samples. The accuracy of the proposed method, as tested on previously unseen data, was 2.80-mm average in needle tip detection and 0.98° in needle trajectory angle. An observer study was designed in which independent annotations by a second observer, blinded to the original observer, were compared with the output of the proposed method. The resultant error was comparable to the measured inter-observer concordance, reinforcing the clinical acceptability of the proposed method. The proposed system has the potential for deployment in clinical routine.

Temporal enhanced ultrasound (TeUS), comprising the analysis of variations in backscattered signals from a tissue over a sequence of ultrasound frames, has been previously proposed as a new paradigm for tissue characterization. In this paper, we propose to use deep recurrent neural networks (RNN) to explicitly model the temporal information in TeUS. By investigating several RNN models, we demonstrate that long short-term memory (LSTM) networks achieve the highest accuracy in separating cancer from benign tissue in the prostate. We also present algorithms for in-depth analysis of LSTM networks. Our in vivo study includes data from 255 prostate biopsy cores of 157 patients. We achieve area under the curve, sensitivity, specificity, and accuracy of 0.96, 0.76, 0.98, and 0.93, respectively. Our result suggests that temporal modeling of TeUS using RNN can significantly improve cancer detection accuracy over previously presented works.

Objectives: Temporal enhanced ultrasound (TeUS) is a new ultrasound-based imaging technique that provides tissue-specific information. Recent studies have shown the potential of TeUS for improving tissue characterization in prostate cancer diagnosis. We study the temporal properties of TeUS-temporal order and length-and present a new framework to assess their impact on tissue information., Methods: We utilize a probabilistic modeling approach using hidden Markov models (HMMs) to capture the temporal signatures of malignant and benign tissues from TeUS signals of nine patients. We model signals of benign and malignant tissues (284 and 286 signals, respectively) in their original temporal order as well as under order permutations. We then compare the resulting models using the Kullback-Liebler divergence and assess their performance differences in characterization. Moreover, we train HMMs using TeUS signals of different durations and compare their model performance when differentiating tissue types., Results: Our findings demonstrate that models of order-preserved signals perform statistically significantly better (85% accuracy) in tissue characterization compared to models of order-altered signals (62% accuracy). The performance degrades as more changes in signal order are introduced. Additionally, models trained on shorter sequences perform as accurately as models of longer sequences., Conclusion: The work presented here strongly indicates that temporal order has substantial impact on TeUS performance; thus, it plays a significant role in conveying tissue-specific information. Furthermore, shorter TeUS signals can relay sufficient information to accurately distinguish between tissue types., Significance: Understanding the impact of TeUS properties facilitates the process of its adopting in diagnostic procedures and provides insights on improving its acquisition.

Objective: A fiber optic vibration sensor is developed and characterized with an ultrawide dynamic sensing range, from less than 1 Hz to clinical ultrasound frequencies near 6 MHz. The vibration sensor consists of a matched pair of fiber Bragg gratings coupled to a custom-built signal processing circuit. The wavelength of a laser diode is locked to one of the many cavity resonances using the Pound-Drever-Hall scheme., Methods: A calibrated piezoelectric vibration element was used to characterize the sensor's strain, temperature, and noise responses. To demonstrate its sensing capability, an ultrasound phantom with built-in low frequency vibration actuation was constructed., Results: The fiber optic senor was shown to simultaneously capture the low frequency vibration and the clinical ultrasound transmission waveforms with nanostrain sensitivity., Conclusion: This miniaturized and sensitive vibration sensor can provide comprehensive information regarding strain response and the resultant ultrasound waveforms.

Temporal-enhanced ultrasound (TeUS) is a novel noninvasive imaging paradigm that captures information from a temporal sequence of backscattered US radio frequency data obtained from a fixed tissue location. This technology has been shown to be effective for classification of various in vivo and ex vivo tissue types including prostate cancer from benign tissue. Our previous studies have indicated two primary phenomena that influence TeUS: 1) changes in tissue temperature due to acoustic absorption and 2) micro vibrations of tissue due to physiological vibration. In this paper, first, a theoretical formulation for TeUS is presented. Next, a series of simulations are carried out to investigate micro vibration as a source of tissue characterizing information in TeUS. The simulations include finite element modeling of micro vibration in synthetic phantoms, followed by US image generation during TeUS imaging. The simulations are performed on two media, a sparse array of scatterers and a medium with pathology mimicking scatterers that match nuclei distribution extracted from a prostate digital pathology data set. Statistical analysis of the simulated TeUS data shows its ability to accurately classify tissue types. Our experiments suggest that TeUS can capture the microstructural differences, including scatterer density, in tissues as they react to micro vibrations.

In recent years, ultrasound has been increasingly used to guide needle insertion procedures for spinal anesthesia. The primary anatomical targets are facet joints and epidural spaces. For these procedures, accurate visualization of the spine anatomy is of critical importance. Challenges arising from the interactions between the ultrasound beam and spinal structures including tilt caused by specular reflections, off-axis interference, and reverberations often result in weakened and blurred vertebra surfaces. Previously, adaptive beamforming methods have been proposed to improve the resolution and contrast. However, most of these methods are not specialized for improving the contrast of specular targets like bones. In this paper, we propose an accumulated angle factor (AAF)-based beamforming method customized for bone surface enhancement. This approach applies a Hilbert transform on delay compensated channel data across the receive aperture. The accumulated phase change across the receive aperture is then calculated and utilized as the weight in the beamforming output. We compared our method with classical delay and sum (DAS) beamforming method and adaptive beamforming methods such as Wiener, phase coherence factor (PCF), CF, and generalized CF (GCF) beamforming. In 12 volunteer data sets, the mean contrast ratio between the vertebrae surface and the surrounding tissue for DAS, Wiener, PCF, CF, GCF, and the proposed AAF methods are 0.49, 0.64, 0.82, 0.77, 0.76, and 0.91, respectively. The contrast is significantly improved in the proposed method.

Accurate identification of the needle target is crucial for effective epidural anesthesia. Currently, epidural needle placement is administered by a manual technique, relying on the sense of feel, which has a significant failure rate. Moreover, misleading the needle may lead to inadequate anesthesia, post dural puncture headaches, and other potential complications. Ultrasound offers guidance to the physician for identification of the needle target, but accurate interpretation and localization remain challenges. A hybrid machine learning system is proposed to automatically localize the needle target for epidural needle placement in ultrasound images of the spine. In particular, a deep network architecture along with a feature augmentation technique is proposed for automatic identification of the anatomical landmarks of the epidural space in ultrasound images. Experimental results of the target localization on planes of 3-D as well as 2-D images have been compared against an expert sonographer. When compared with the expert annotations, the average lateral and vertical errors on the planes of 3-D test data were 1 and 0.4 mm, respectively. On 2-D test data set, an average lateral error of 1.7 mm and vertical error of 0.8 mm were acquired.

Echocardiography (echo) is a skilled technical procedure that depends on the experience of the operator. The aim of this paper is to reduce user variability in data acquisition by automatically computing a score of echo quality for operator feedback. To do this, a deep convolutional neural network model, trained on a large set of samples, was developed for scoring apical four-chamber (A4C) echo. In this paper, 6,916 end-systolic echo images were manually studied by an expert cardiologist and were assigned a score between 0 (not acceptable) and 5 (excellent). The images were divided into two independent training-validation and test sets. The network architecture and its parameters were based on the stochastic approach of the particle swarm optimization on the training-validation data. The mean absolute error between the scores from the ultimately trained model and the expert's manual scores was 0.71 ± 0.58. The reported error was comparable to the measured intra-rater reliability. The learned features of the network were visually interpretable and could be mapped to the anatomy of the heart in the A4C echo, giving confidence in the training result. The computation time for the proposed network architecture, running on a graphics processing unit, was less than 10 ms per frame, sufficient for real-time deployment. The proposed approach has the potential to facilitate the widespread use of echo at the point-of-care and enable early and timely diagnosis and treatment. Finally, the approach did not use any specific assumptions about the A4C echo, so it could be generalizable to other standard echo views.

We propose a joint information approach for automatic analysis of 2D echocardiography (echo) data. The approach combines a priori images, their segmentations and patient diagnostic information within a unified framework to determine various clinical parameters, such as cardiac chamber volumes, and cardiac disease labels. The main idea behind the approach is to employ joint Independent Component Analysis of both echo image intensity information and corresponding segmentation labels to generate models that jointly describe the image and label space of echo patients on multiple apical views, instead of independently. These models are then both used for segmentation and volume estimation of cardiac chambers such as the left atrium and for detecting pathological abnormalities such as mitral regurgitation. We validate the approach on a large cohort of echoes obtained from 6,993 studies. We report performance of the proposed approach in estimation of the left-atrium volume and detection of mitral-regurgitation severity. A correlation coefficient of 0.87 was achieved for volume estimation of the left atrium when compared to the clinical report. Moreover, we classified patients that suffer from moderate or severe mitral regurgitation with an average accuracy of 82%.

Segmentation of the wrist bones in CT images has been frequently used in different clinical applications including arthritis evaluation, bone age assessment and image-guided interventions. The major challenges include non-uniformity and spongy textures of the bone tissue as well as narrow inter-bone spaces. In this work, we propose an automatic wrist bone segmentation technique for CT images based on a statistical model that captures the shape and pose variations of the wrist joint across 60 example wrists at nine different wrist positions. To establish the correspondences across the training shapes at neutral positions, the wrist bone surfaces are jointly aligned using a group-wise registration framework based on a Gaussian Mixture Model. Principal component analysis is then used to determine the major modes of shape variations. The variations in poses not only across the population but also across different wrist positions are incorporated in two pose models. An intra-subject pose model is developed by utilizing the similarity transforms at all wrist positions across the population. Further, an inter-subject pose model is used to model the pose variations across different wrist positions. For segmentation of the wrist bones in CT images, the developed model is registered to the edge point cloud extracted from the CT volume through an expectation maximization based probabilistic approach. Residual registration errors are corrected by application of a non-rigid registration technique. We validate the proposed segmentation method by registering the wrist model to a total of 66 unseen CT volumes of average voxel size of 0.38 mm. We report a mean surface distance error of 0.33 mm and a mean Jaccard index of 0.86.

Low-dose-rate prostate brachytherapy treatment takes place by implantation of small radioactive seeds in and sometimes adjacent to the prostate gland. A patient specific target anatomy for seed placement is usually determined by contouring a set of collected transrectal ultrasound images prior to implantation. Standard-of-care in prostate brachytherapy is to delineate the clinical target anatomy, which closely follows the real prostate boundary. Subsequently, the boundary is dilated with respect to the clinical guidelines to determine a planning target volume. Manual contouring of these two anatomical targets is a tedious task with relatively high observer variability. In this work, we aim to reduce the segmentation variability and planning time by proposing an efficient learning-based multi-label segmentation algorithm. We incorporate a sparse representation approach in our methodology to learn a dictionary of sparse joint elements consisting of images, and clinical and planning target volume segmentation. The generated dictionary inherently captures the relationships among elements, which also incorporates the institutional clinical guidelines. The proposed multi-label segmentation method is evaluated on a dataset of 590 brachytherapy treatment records by 5-fold cross validation. We show clinically acceptable instantaneous segmentation results for both target volumes.

A common challenge when performing surface-based registration of images is ensuring that the surfaces accurately represent consistent anatomical boundaries. Image segmentation may be difficult in some regions due to either poor contrast, low slice resolution, or tissue ambiguities. To address this, we present a novel non-rigid surface registration method designed to register two partial surfaces, capable of ignoring regions where the anatomical boundary is unclear. Our probabilistic approach incorporates prior geometric information in the form of a statistical shape model (SSM), and physical knowledge in the form of a finite element model (FEM). We validate results in the context of prostate interventions by registering pre-operative magnetic resonance imaging (MRI) to 3D transrectal ultrasound (TRUS). We show that both the geometric and physical priors significantly decrease net target registration error (TRE), leading to TREs of 2.35 ± 0.81 mm and 2.81 ± 0.66 mm when applied to full and partial surfaces, respectively. We investigate robustness in response to errors in segmentation, varying levels of missing data, and adjusting the tunable parameters. Results demonstrate that the proposed surface registration method is an efficient, robust, and effective solution for fusing data from multiple modalities.

Unlabelled: This paper presents the results of a computer-aided intervention solution to demonstrate the application of RF time series for characterization of prostate cancer, in vivo., Methods: We pre-process RF time series features extracted from 14 patients using hierarchical clustering to remove possible outliers. Then, we demonstrate that the mean central frequency and wavelet features extracted from a group of patients can be used to build a nonlinear classifier which can be applied successfully to differentiate between cancerous and normal tissue regions of an unseen patient., Results: In a cross-validation strategy, we show an average area under receiver operating characteristic curve (AUC) of 0.93 and classification accuracy of 80%. To validate our results, we present a detailed ultrasound to histology registration framework., Conclusion: Ultrasound RF time series results in differentiation of cancerous and normal tissue with high AUC.

In surface-based registration for image-guided interventions, the presence of missing data can be a significant issue. This often arises with real-time imaging modalities such as ultrasound, where poor contrast can make tissue boundaries difficult to distinguish from surrounding tissue. Missing data poses two challenges: ambiguity in establishing correspondences; and extrapolation of the deformation field to those missing regions. To address these, we present a novel non-rigid registration method. For establishing correspondences, we use a probabilistic framework based on a Gaussian mixture model (GMM) that treats one surface as a potentially partial observation. To extrapolate and constrain the deformation field, we incorporate biomechanical prior knowledge in the form of a finite element model (FEM). We validate the algorithm, referred to as GMM-FEM, in the context of prostate interventions. Our method leads to a significant reduction in target registration error (TRE) compared to similar state-of-the-art registration algorithms in the case of missing data up to 30%, with a mean TRE of 2.6 mm. The method also performs well when full segmentations are available, leading to TREs that are comparable to or better than other surface-based techniques. We also analyze robustness of our approach, showing that GMM-FEM is a practical and reliable solution for surface-based registration.

The objective of sensorless freehand 3-D ultrasound imaging is to eliminate the need for additional tracking hardware and reduce cost and complexity. However, the accuracy of current out-of-plane pose estimation is main obstacle for full 6-degree-of-freedom (DoF) tracking. We propose a new filter-based speckle tracking framework to increase the accuracy of out-of-plane displacement estimation. In this framework, we use the displacement estimation not only for the specific speckle pattern, but for the entire image. We develop a nonlocal means (NLM) filter based on a probabilistic normal variance mixture model of ultrasound, known as Rician-inverse Gaussian (RiIG). To aggregate the local displacement estimations, Stein's unbiased risk estimate (SURE) is used as a quality measure of the estimations. We derive an explicit analytical form of SURE for the RiIG model and use it as a weight factor. The proposed filter-based speckle tracking framework is formulated and evaluated for three commonly used noise models, including the RiIG model. The out-of-plane estimations are compared with our previously proposed model-based algorithm in a set of ex vivo experiments for different tissue types. We show that the proposed RiIG filter-based method is more accurate and less tissue-dependent than the other methods. The proposed method is also evaluated in vivo on the spines of five different subjects to assess the feasibility of a clinical application. The 6-DoF transform parameters are estimated and compared with the electromagnetic tracker measurements. The results show higher tracking accuracy for typical small lateral displacements and tilt rotations between image pairs.

Objective: This paper presents the results of a new approach for selection of RF time series features based on joint independent component analysis for in vivo characterization of prostate cancer., Methods: We project three sets of RF time series features extracted from the spectrum, fractal dimension, and the wavelet transform of the ultrasound RF data on a space spanned by five joint independent components. Then, we demonstrate that the obtained mixing coefficients from a group of patients can be used to train a classifier, which can be applied to characterize cancerous regions of a test patient., Results: In a leave-one-patient-out cross validation, an area under receiver operating characteristic curve of 0.93 and classification accuracy of 84% are achieved., Conclusion: Ultrasound RF time series can be used to accurately characterize prostate cancer, in vivo without the need for exhaustive search in the feature space., Significance: We use joint independent component analysis for systematic fusion of multiple sets of RF time series features, within a machine learning framework, to characterize PCa in an in vivo study.

Low-dose-rate brachytherapy is a radiation treatment method for localized prostate cancer. The standard of care for this treatment procedure is to acquire transrectal ultrasound images of the prostate in order to devise a plan to deliver sufficient radiation dose to the cancerous tissue. Brachytherapy planning involves delineation of contours in these images, which closely follow the prostate boundary, i.e., clinical target volume. This process is currently performed either manually or semi-automatically, which requires user interaction for landmark initialization. In this paper, we propose a multi-atlas fusion framework to automatically delineate the clinical target volume in ultrasound images. A dataset of a priori segmented ultrasound images, i.e., atlases, is registered to a target image. We introduce a pairwise atlas agreement factor that combines an image-similarity metric and similarity between a priori segmented contours. This factor is used in an atlas selection algorithm to prune the dataset before combining the atlas contours to produce a consensus segmentation. We evaluate the proposed segmentation approach on a set of 280 transrectal prostate volume studies. The proposed method produces segmentation results that are within the range of observer variability when compared to a semi-automatic segmentation technique that is routinely used in our cancer clinic.

This work reports the use of ultrasound radio frequency (RF) time series analysis as a method for ultrasound-based classification of malignant breast lesions. The RF time series method is versatile and requires only a few seconds of raw ultrasound data with no need for additional instrumentation. Using the RF time series features, and a machine learning framework, we have generated malignancy maps, from the estimated cancer likelihood, for decision support in biopsy recommendation. These maps depict the likelihood of malignancy for regions of size 1 mm(2) within the suspicious lesions. We report an area under receiver operating characteristics curve of 0.86 (95% confidence interval [CI]: 0.84%-0.90%) using support vector machines and 0.81 (95% CI: 0.78-0.85) using Random Forests classification algorithms, on 22 subjects with leave-one-subject-out cross-validation. Changing the classification method yielded consistent results which indicates the robustness of this tissue typing method. The findings of this report suggest that ultrasound RF time series, along with the developed machine learning framework, can help in differentiating malignant from benign breast lesions, subsequently reducing the number of unnecessary biopsies after mammography screening.

Most conventional spine interventions are performed under X-ray fluoroscopy guidance. In recent years, there has been a growing interest to develop nonionizing imaging alternatives to guide these procedures. Ultrasound guidance has emerged as a leading alternative. However, a challenging problem is automatic identification of the spinal anatomy in ultrasound data. In this paper, we propose a local phase-based bone feature enhancement technique that can robustly identify the spine surface in ultrasound images. The local phase information is obtained using a gradient energy tensor filter. This information is used to construct local phase tensors in ultrasound images, which highlight the spine surface. We show that our proposed approach results in a more distinct enhancement of the bone surfaces compared to recently proposed techniques based on monogenic scale-space filters and logarithmic Gabor filters. We also demonstrate that registration accuracy of a statistical shape+pose model of the spine to 3-D ultrasound images can be significantly improved, using the proposed method, compared to those obtained using monogenic scale-space filters and logarithmic Gabor filters.

A big challenge in sensorless image-based ultrasound tracking is in the out-of-plane motion estimation. The correlation value of a specific model of speckle known as fully developed speckle (FDS) can be used to estimate the out-of-plane displacement. In real tissue, this kind of pattern is rare and the deviation of speckle pattern from the ideal FDS model diminishes the accuracy of the out-of-plane motion estimation. In this paper a new method for estimation of the out-of-plane motion is proposed. Firstly a closed-form mathematical derivation is provided for the correlation of two RF echo signal patches at different positions. A linear regression model of the ultrasound beam profile is proposed to account for the spatial variability of the ultrasound beam and enhance the accuracy of out-of-plane motion estimation in real tissue. The statistical model of speckle used here is based on the Rician-Inverse Gaussian (RiIG) stochastic process of the speckle formation, which can be considered as a generalized form of the K-distribution with richer parametrization. In this work, for the first time the second-order statistics of the RIG model is used for speckle tracking. This statistical model allows for derivation of a closed-form formulation for the correlation coefficient based on the statistical parameters of every patch. Since the effect of coherency is considered in the RiIG model, it increases the reliability of the out-of-plane motion estimation. The flexibility of the proposed method enables almost any patch through the whole image to be used for the purpose of displacement estimation. The method has been evaluated both on ex vivo and in vivo tissues in various experiments including out-of-plane rotation (tilt, yaw) and free-hand imaging. The overall outcome demonstrates the potential of the proposed method for in vivo tissues.

This paper presents the prototype for an augmented reality haptic simulation system with potential for spinal needle insertion training. The proposed system is composed of a torso mannequin, a MicronTracker2 optical tracking system, a PHANToM haptic device, and a graphical user interface to provide visual feedback. The system allows users to perform simulated needle insertions on a physical mannequin overlaid with an augmented reality cutaway of patient anatomy. A tissue model based on a finite-element model provides force during the insertion. The system allows for training without the need for the presence of a trained clinician or access to live patients or cadavers. A pilot user study demonstrates the potential and functionality of the system.

Segmentation of the spinal column from computed tomography (CT) images is a preprocessing step for a range of image-guided interventions. One intervention that would benefit from accurate segmentation is spinal needle injection. Previous spinal segmentation techniques have primarily focused on identification and separate segmentation of each vertebra. Recently, statistical multi-object shape models have been introduced to extract common statistical characteristics between several anatomies. These models can be used for segmentation purposes because they are robust, accurate, and computationally tractable. In this paper, we develop a statistical multi-vertebrae shape+pose model and propose a novel registration-based technique to segment the CT images of spine. The multi-vertebrae statistical model captures the variations in shape and pose simultaneously, which reduces the number of registration parameters. We validate our technique in terms of accuracy and robustness of multi-vertebrae segmentation of CT images acquired from lumbar vertebrae of 32 subjects. The mean error of the proposed technique is below 2 mm, which is sufficient for many spinal needle injection procedures, such as facet joint injections.

We propose an augmented reality system to identify lumbar vertebral levels to assist in spinal needle insertion for epidural anesthesia. These procedures require careful placement of a needle to ensure effective delivery of anesthetics and to avoid damaging sensitive tissue such as nerves. In this system, a trinocular camera tracks an ultrasound transducer during the acquisition of a sequence of B-mode images. The system generates an ultrasound panorama image of the lumbar spine, automatically identifies the lumbar levels in the panorama image, and overlays the identified levels on a live camera view of the patient's back. Validation is performed to test the accuracy of panorama generation, lumbar level identification, overall system accuracy, and the effect of changes in the curvature of the spine during the examination. The results from 17 subjects demonstrate the feasibility and capability of achieving an error within clinically acceptable range for epidural anaesthesia.

Robot-assisted laparoscopic radical prostatectomy (RALRP) using the da Vinci surgical system is the current state-of-the-art treatment option for clinically confined prostate cancer. Given the limited field of view of the surgical site in RALRP, several groups have proposed the integration of transrectal ultrasound (TRUS) imaging in the surgical workflow to assist with accurate resection of the prostate and the sparing of the neurovascular bundles (NVBs). We previously introduced a robotic TRUS manipulator and a method for automatically tracking da Vinci surgical instruments with the TRUS imaging plane, in order to facilitate the integration of intraoperative TRUS in RALRP. Rapid and automatic registration of the kinematic frames of the da Vinci surgical system and the robotic TRUS probe manipulator is a critical component of the instrument tracking system. In this paper, we propose a fully automatic registration technique based on automatic 3-D TRUS localization of robot instrument tips pressed against the air-tissue boundary anterior to the prostate. The detection approach uses a multiscale filtering technique to identify and localize surgical instrument tips in the TRUS volume, and could also be used to detect other surface fiducials in 3-D ultrasound. Experiments have been performed using a tissue phantom and two ex vivo tissue samples to show the feasibility of the proposed methods. Also, an initial in vivo evaluation of the system has been carried out on a live anaesthetized dog with a da Vinci Si surgical system and a target registration error (defined as the root mean square distance of corresponding points after registration) of 2.68 mm has been achieved. Results show this method's accuracy and consistency for automatic registration of TRUS images to the da Vinci surgical system.

This paper presents the results of a feasibility study to demonstrate the application of ultrasound RF time series imaging to accurately differentiate ablated and nonablated tissue. For 12 ex vivo and two in situ tissue samples, RF ultrasound signals are acquired prior to, and following, high-intensity ultrasound ablation. Spatial and temporal features of these signals are used to characterize ablated and nonablated tissue in a supervised-learning framework. In cross-validation evaluation, a subset of four features extracted from RF time series produce a classification accuracy of 84.5%, an area under ROC curve of 0.91 for ex vivo data, and an accuracy of 85% for in situ data. Ultrasound RF time series is a promising approach for characterizing ablated tissue.

Ultrasound (US) radio-frequency (RF) time series is an effective tissue classification method that enables accurate cancer diagnosis, but the mechanisms underlying this method are not completely understood. This paper presents a model to describe the variations in tissue temperature and sound speed that take place during the RF time series scanning procedures and relate these variations to US backscattering. The model was used to derive four novel characterization features. These features were used to classify three animal tissues, and they obtained accuracies as high as 88.01%. The performance of the proposed features was compared with RF time series features proposed in a previous study. The results indicated that the US-induced variations in tissue temperature and sound speed, which were used to derive the proposed features, were important contributors to the tissue typing capabilities of the RF time series. Simulations carried out to estimate the heating induced during the scanning procedure employed in this study showed temperature rises lower than 2 °C. The model and results presented in this paper can be used to improve the RF time series.

This paper presents a novel, fast, group-wise registration technique based on establishing soft correspondences between groups of point sets. The registration approach is used to create a statistical shape model, capable of learning the shape variations within a training set. The shape model consists of a mean shape and its transformations to all training shapes. We decouple the procedure into two steps: updating the mean shape and registering it to the training shapes. The algorithm alternates between these two steps until convergence. Following the generation of the statistical shape model, we use the soft correspondence approach to register the model to a new observation. We perform extensive experiments on two data sets: lumbar spine and hippocampi. We compare our algorithm to available state-of- the-art group-wise registration algorithms including feature-based and volume-based approaches. We demonstrate improved generalization, specificity and compactness compared to these algorithms.

Purpose: Ultrasound (US) guidance in facet joint injections has been reported previously as an alternative to imaging modalities with ionizing radiation. However, this technique has not been adopted in the clinical routine, due to difficulties in the visualization of the target joint in US and simultaneous manipulation of the needle., Methods: We propose a technique to increase targeting accuracy and efficiency in facet joint injections. This is achieved by electromagnetically tracking the positions of the US transducer and the needle, and recording tracked US snapshots (TUSS). The needle is navigated using the acquired US snapshots., Results: In cadaveric lamb model, the success rate of facet joint injections by five orthopedic surgery residents significantly increased from 44.4% with freehand US guidance to 93.3% with TUSS guidance. Needle insertion time significantly decreased from 47.9 ± 34.2 s to 36.1 ± 28.7 s (mean ± SD). In a synthetic human spine model, a success rate of 96.7% was achieved with TUSS. The targeting accuracy of the presented system in a gel phantom was 1.03 ± 0.48 mm (mean ± SD)., Conclusion: Needle guidance with TUSS improves the success rate and time efficiency in spinal facet joint injections. This technique readily translates also to other spinal needle placement applications.

This paper presents a multiple-object 2-D-3-D registration technique for noninvasively identifying the poses of fracture fragments in the space of a preoperative treatment plan. The plan is made by manipulating and aligning computer models of individual fracture fragments that are segmented from a diagnostic computed tomography. The registration technique iteratively updates the treatment plan and matches its digitally reconstructed radiographs to a small number of intraoperative fluoroscopic images. The proposed approach combines an image similarity metric that integrates edge information with mutual information, and a global-local optimization scheme, to deal with challenges associated with the registration of multiple small fragments and limited imaging orientations in the operating room. The method is easy to use as minimum user interaction is required. Experiments on simulated fractures and two distal radius fracture phantoms demonstrate clinically acceptable target registration errors with capture range as large as 10 mm.

Image registration is a single point of failure in the image-guided computer-assisted surgery. Registration is primarily used to align and fuse the data sets taken from patient's anatomy before and during surgeries. Point-based rigid-body registration is usually performed by identifying corresponding fiducials (either natural landmarks or implanted ones) in the data sets. Since the localization of fiducials is imprecise and is generally perturbed by random noise, the performed registration is imperfect and has some error. Previous work has extensively analyzed the behavior of this error when the fiducial localization error has zero-mean over the entire set of fiducials. However, if noise has a nonzero-mean or a bias, no formulation yet exists to determine the effect of noise on the overall registration accuracy. In this work, we derive novel formulations that relate the bias in the localized fiducials to the accuracy of the performed registration. We analytically and numerically demonstrate that by eliminating the estimated bias from the measured fiducial locations, one can effectively increase the accuracy of the performed registration.

We study the effect of point dependent (heteroscedastic) and identically distributed anisotropic fiducial localization noise on fiducial target registration error (TRE). We derive an analytic expression, based on the concept of mechanism spatial stiffness, for predicting TRE. The accuracy of the predicted TRE is compared to simulated values where the optimal registration transformation is computed using the heteroscedastic errors in variables algorithm. The predicted values are shown to be contained by the 95% confidence intervals of the root mean square TRE obtained from the simulations.

Rigid-body point-based registration is frequently used in computer assisted surgery to align corresponding points, or fiducials, in preoperative and intraoperative data. This alignment is mostly achieved by assuming the same homogeneous error distribution for all the points; however, due to the properties of the medical instruments used in measuring the point coordinates, the error distribution might be inhomogeneous and different for each point. In this paper, in an effort to understand the effect of error distribution in the localized points on the performed registration, we derive a closed-form solution relating the error distribution of each point with the performed registration accuracy. The solution uses maximum likelihood estimation to calculate the probability density function of registration error at each fiducial point. Extensive numerical simulations are performed to validated the proposed solution.

We propose a novel and accurate method based on ultrasound RF time series analysis and an extended version of support vector machine classification for generating probabilistic cancer maps that can augment ultrasound images of prostate and enhance the biopsy process. To form the RF time series, we record sequential ultrasound RF echoes backscattered from tissue while the imaging probe and the tissue are stationary in position. We show that RF time series acquired from agar-gelatin-based tissue mimicking phantoms, with difference only in the size of cell-mimicking microscopic glass beads, are distinguishable with statistically reliable accuracies up to 80.5%. This fact indicates that the differences in tissue microstructures affect the ultrasound RF time series features. Based on this phenomenon, in an ex vivo study involving 35 prostate specimens, we show that the features extracted from RF time series are significantly more accurate and sensitive compared to two other established categories of ultrasound-based tissue typing methods. We report an area under receiver operating characteristic curve of 0.95 in tenfold cross validation and 0.82 in leave-one-patient-out cross validation for detection of prostate cancer.

In point-based rigid-body registration, target registration error (TRE) is an important measure of the accuracy of the performed registration. The registration's accuracy depends on the fiducial localization error (FLE) which, in turn, is due to the measurement errors in the points (fiducials) used to perform the registration. FLE may have different characteristics and distributions at each point of the registering data sets, and along each orthogonal axis. Previously, the distribution of TRE was estimated based on the assumption that FLE has an independent, identical, and isotropic or anisotropic distribution for each point in the registering data sets. In this article, we present a general solution based on the Maximum Likelihood (ML) algorithm that estimates the distribution of TRE for the cases where FLE has an independent, identical or inhomogeneous, isotropic or anisotropic, distribution at each point in the registering data sets, and when an algorithm is available that is capable of calculating the optimum registration to first order. Mathematically, we show that the proposed algorithm simplifies to the one proposed by Fitzpatrick and West when FLE has an independent, identical, and isotropic distribution in the registering data sets. Furthermore, we use numerical simulations to show that the proposed algorithm accurately estimates the distribution of TRE when FLE has an independent, inhomogeneous, and anisotropic distribution in the registering data sets.

Speckle noise is an inherent nature of ultrasound images, which may have negative effect on image interpretation and diagnostic tasks. In this paper, we propose several multiscale nonlinear thresholding methods for ultrasound speckle suppression. The wavelet coefficients of the logarithm of image are modeled as the sum of a noise-free component plus an independent noise. Assuming that the noise-free component has some local mixture distribution (MD), and the noise is either Gaussian or Rayleigh, we derive the minimum mean squared error (MMSE) and the averaged maximum a posteriori (AMAP) estimators for noise reduction. We use Gaussian and Laplacian MD for each noise-free wavelet coefficient to characterize their heavy-tailed property. Since we estimate the parameters of the MD using the expectation maximization (EM) algorithm and local neighbors, the proposed MD incorporates some information about the intrascale dependency of the wavelet coefficients. To evaluate our spatially adaptive despeckling methods, we use both real medical ultrasound and synthetically introduced speckle images for speckle suppression. The simulation results show that our method outperforms several recently and the state-of-the-art techniques qualitatively and quantitatively.

This paper presents a framework for the design of a haptic-based medical ultrasound training simulator. The proposed simulator is composed of a PHANToM haptic device and a modular software package that allows for visual feedback and kinesthetic interactions between an operator and multimodality image databases. The system provides real-time ultrasound images in the same fashion as a typical ultrasound machine, enhanced with corresponding augmented computerized tomographic (CT) and/or MRI images. The proposed training system allows trainees to develop radiology techniques and knowledge of the patient's anatomy with minimum practice on live patients, or in places or at times when radiology devices or patients with rare cases may not be available. Low-level details of the software structure that can be migrated to other similar medical simulators are described. A preliminary human factors study, conducted on the prototype of the developed simulator, demonstrates the potential usage of the system for clinical training.

We present and validate a novel registration algorithm mapping two data sets, generated from a rigid object, in the presence of Gaussian noise. The proposed method is based on the Unscented Kalman Filter (UKF) algorithm that is generally employed for analyzing nonlinear systems corrupted by additive Gaussian noise. First, we employ our proposed registration algorithm to fit two randomly generated data sets in the presence of isotropic Gaussian noise, when the corresponding points between the two data sets are assumed to be known. Then, we extend the registration method to the case where the data (with known correspondences) is stimulated by anisotropic Gaussian noise. The new registration method not only reliably converges to the correct registration solution, but it also estimates the variance, as a confidence measure, for each of the estimated registration transformation parameters. Furthermore, we employ the proposed registration algorithm for rigid-body, point-based registration where corresponding points between two registering data sets are unknown. The algorithm is tested on point data sets which are garnered from a pelvic cadaver and a scaphoid bone phantom by means of computed tomography (CT) and tracked free-hand ultrasound imaging. The collected 3-D points in the ultrasound frame are registered to the 3-D meshes in the CT frame by using the proposed and the standard Iterative Closest Points (ICP) registration algorithms. Experimental results demonstrate that our proposed method significantly outperforms the ICP registration algorithm in the presence of additive Gaussian noise. It is also shown that the proposed registration algorithm is more robust than the ICP registration algorithm in terms of outliers in data sets and initial misalignment between the two data sets.