Search

Significance: Polyp size is important for selecting the surveillance interval or treatment policy. Nevertheless, it is challenging to accurately estimate the polyp size during endoscopy. An easy and cost-effective function to assist in polyp size estimation is required. Aim: To propose a virtual scale function for endoscopy and evaluate its performance and expected accuracy. Approach: An adaptive virtual scale behavior was demonstrated. The measurement error of the virtual scale along the distance between the tip of the endoscope and the object plane was evaluated using graph paper. The accuracy of polyp size estimation by an expert endoscopist was compared with the accuracy of the biopsy forceps method using phantom images. Results: The measurement errors of the virtual scale were ≤ 0.7 mm when the distance to the graph paper, which faced the tip of the endoscope, varied from 4 to 30 mm. The accuracy with the virtual scale was significantly higher than that obtained with biopsy forceps (5.3 ± 5.5 % versus 11.9 ± 9.4 % , P

According to the World Health Organization (WHO), novel coronavirus (COVID-19) is an infectious disease and has a significant social and economic impact. The main challenge in fighting against this disease is its scale. Due to the outbreak, medical facilities are under pressure due to case numbers. A quick diagnosis system is required to address these challenges. To this end, a stochastic deep learning model is proposed. The main idea is to constrain the deep-representations over a Gaussian prior to reinforce the discriminability in feature space. The model can work on chest X-ray or CT-scan images. It provides a fast diagnosis of COVID-19 and can scale seamlessly. The work presents a comprehensive evaluation of previously proposed approaches for X-ray based disease diagnosis. The approach works by learning a latent space over X-ray image distribution from the ensemble of state-of-the-art convolutional-nets, and then linearly regressing the predictions from an ensemble of classifiers which take the latent vector as input. We experimented with publicly available datasets having three classes: COVID-19, normal and pneumonia yielding an overall accuracy and AUC of 0.91 and 0.97, respectively. Moreover, for robust evaluation, experiments were performed on a large chest X-ray dataset to classify among Atelectasis, Effusion, Infiltration, Nodule, and Pneumonia classes. The results demonstrate that the proposed model has better understanding of the X-ray images which make the network more generic to be later used with other domains of medical image analysis.

Background Margin reflex distance 1 (MRD1), margin reflex distance 2 (MRD2), and levator muscle function (LF) are crucial metrics for ptosis evaluation and management. However, manual measurements of MRD1, MRD2, and LF are time-consuming, subjective, and prone to human error. Smartphone-based artificial intelligence (AI) image processing is a potential solution to overcome these limitations. Objective We propose the first smartphone-based AI-assisted image processing algorithm for MRD1, MRD2, and LF measurements. Methods This observational study included 822 eyes of 411 volunteers aged over 18 years from August 1, 2020, to April 30, 2021. Six orbital photographs (bilateral primary gaze, up-gaze, and down-gaze) were taken using a smartphone (iPhone 11 Pro Max). The gold-standard measurements and normalized eye photographs were obtained from these orbital photographs and compiled using AI-assisted software to create MRD1, MRD2, and LF models. Results The Pearson correlation coefficients between the gold-standard measurements and the predicted values obtained with the MRD1 and MRD2 models were excellent (r=0.91 and 0.88, respectively) and that obtained with the LF model was good (r=0.73). The intraclass correlation coefficient demonstrated excellent agreement between the gold-standard measurements and the values predicted by the MRD1 and MRD2 models (0.90 and 0.84, respectively), and substantial agreement with the LF model (0.69). The mean absolute errors were 0.35 mm, 0.37 mm, and 1.06 mm for the MRD1, MRD2, and LF models, respectively. The 95% limits of agreement were –0.94 to 0.94 mm for the MRD1 model, –0.92 to 1.03 mm for the MRD2 model, and –0.63 to 2.53 mm for the LF model. Conclusions We developed the first smartphone-based AI-assisted image processing algorithm for eyelid measurements. MRD1, MRD2, and LF measures can be taken in a quick, objective, and convenient manner. Furthermore, by using a smartphone, the examiner can check these measurements anywhere and at any time, which facilitates data collection.

Lung segmentation is a key step of thoracic computed tomography (CT) image processing, and it plays an important role in computer-aided pulmonary disease diagnostics. However, the presence of image noises, pathologies, vessels, individual anatomical varieties, and so on makes lung segmentation a complex task. In this paper, we present a fully automatic algorithm for segmenting lungs from thoracic CT images accurately. An input image is first spilt into a set of non-overlapping fixed-sized image patches, and a deep convolutional neural network model is constructed to extract initial lung regions by classifying image patches. Superpixel segmentation is then performed on the preprocessed thoracic CT image, and the lung contours are locally refined according to corresponding superpixel contours with our adjacent point statistics method. Segmented lung contours are further globally refined by an edge direction tracing technique for the inclusion of juxta-pleural lesions. Our algorithm is tested on a group of thoracic CT scans with interstitial lung diseases. Experiments show that our algorithm creates an average Dice similarity coefficient of 97.95% and Jaccard's similarity index of 94.48%, with 2.8% average over-segmentation rate and 3.3% under-segmentation rate compared with manually segmented results. Meanwhile, it shows better performance compared with several feature-based machine learning methods and current methods on lung segmentation.

Significance: Speckle noise limits the diagnostic capabilities of optical coherence tomography (OCT) images, causing both a reduction in contrast and a less accurate assessment of the microstructural morphology of the tissue. Aim: We present a speckle-noise reduction method for OCT volumes that exploits the advantages of adaptive-noise wavelet thresholding with a wavelet compounding method applied to several frames acquired from consecutive positions. The method takes advantage of the wavelet representation of the speckle statistics, calculated properly from a homogeneous sample or a region of the noisy volume. Approach: The proposed method was first compared quantitatively with different state-of-the-art approaches by being applied to three different clinical dermatological OCT volumes with three different OCT settings. The method was also applied to a public retinal spectral-domain OCT dataset to demonstrate its applicability to different imaging modalities. Results: The results based on four different metrics demonstrate that the proposed method achieved the best performance among the tested techniques in suppressing noise and preserving structural information. Conclusions: The proposed OCT denoising technique has the potential to adapt to different image OCT settings and noise environments and to improve image quality prior to clinical diagnosis based on visual assessment.

Motivation Study of brain images of rodent animals is the most straightforward way to understand brain functions and neural basis of physiological functions. An important step in brain image analysis is to precisely assign signal labels to specified brain regions through matching brain images to standardized brain reference atlases. However, no significant effort has been made to match different types of brain images to atlas images due to influence of artifact operation during slice preparation, relatively low resolution of images and large structural variations in individual brains. Results In this study, we develop a novel image sequence matching procedure, termed accurate and robust matching brain image sequences (ARMBIS), to match brain image sequences to established atlas image sequences. First, for a given query image sequence a scaling factor is estimated to match a reference image sequence by a curve fitting algorithm based on geometric features. Then, the texture features as well as the scale and rotation invariant shape features are extracted, and a dynamic programming-based procedure is designed to select optimal image subsequences. Finally, a hierarchical decision approach is employed to find the best matched subsequence using regional textures. Our simulation studies show that ARMBIS is effective and robust to image deformations such as linear or non-linear scaling, 2D or 3D rotations, tissue tear and tissue loss. We demonstrate the superior performance of ARMBIS on three types of brain images including magnetic resonance imaging, mCherry with 4′,6-diamidino-2-phenylindole (DAPI) staining and green fluorescent protein without DAPI staining images. Availability and implementation The R software package is freely available at https://www.synapse.org/#!Synapse:syn18638510/wiki/591054 for Not-For-Profit Institutions. If you are a For-Profit Institution, please contact the corresponding author. Supplementary information Supplementary data are available at Bioinformatics online.

Significance: Digital holographic microscopy (DHM) is a promising technique for the study of semitransparent biological specimen such as red blood cells (RBCs). It is important and meaningful to detect and count biological cells at the single cell level in biomedical images for biomarker discovery and disease diagnostics. However, the biological cell analysis based on phase information of images is inefficient due to the complexity of numerical phase reconstruction algorithm applied to raw hologram images. New cell study methods based on diffraction pattern directly are desirable. Aim: Deep fully convolutional networks (FCNs) were developed on raw hologram images directly for high-throughput label-free cell detection and counting to assist the biological cell analysis in the future. Approach: The raw diffraction patterns of RBCs were recorded by use of DHM. Ground-truth mask images were labeled based on phase images reconstructed from RBC holograms using numerical reconstruction algorithm. A deep FCN, which is UNet, was trained on the diffraction pattern images to achieve the label-free cell detection and counting. Results: The implemented deep FCNs provide a promising way to high-throughput and label-free counting of RBCs with a counting accuracy of 99% at a throughput rate of greater than 288 cells per second and 200 μm×200 μm field of view at the single cell level. Compared to convolutional neural networks, the FCNs can get much better results in terms of accuracy and throughput rate. Conclusions: High-throughput label-free cell detection and counting were successfully achieved from diffraction patterns with deep FCNs. It is a promising approach for biological specimen analysis based on raw hologram directly.

Significance: Large space-bandwidth product is highly desirable in many biomedical imaging. Fourier ptychographic microscopy (FPM) is a computational imaging technique that can significantly increase the space-bandwidth product of a standard microscope. The illuminator of a Fourier ptychographic microscope is not flexible at present, and it is inconvenient to meet different imaging needs. Aim: An illuminator based on a two-axis motorized rotation stage was presented to provide a more flexible illuminating way with the goal of meeting different imaging needs. Approach: The illuminator adopts a concentric illuminating method to provide coherent illumination in any direction on the sample plane. The sampling pattern can be freely designed and changed according to the parameters of the imaging system. A dither removing algorithm was proposed to remove the potential dither influence introduced in the image acquisition process. Results: The illuminator could be conveniently integrated into different imaging systems. The feasibility and flexibility were demonstrated by applying it to imaging systems with numerical aperture of 0.045 and 0.01. The resolution gain is about 4- and 13-fold, respectively. The effectiveness of the dither removing algorithm was validated in both simulation and experiment. Conclusions: A more flexible illuminator for FPM was presented to meet different imaging needs. A dither removing algorithm was proposed to remove dither influence.

Motivation Automatic alignment, especially fiducial marker-based alignment, has become increasingly important due to the high demand of subtomogram averaging and the rapid development of large-field electron microscopy. Among the alignment steps, fiducial marker tracking is a crucial one that determines the quality of the final alignment. Yet, it is still a challenging problem to track the fiducial markers accurately and effectively in a fully automatic manner. Results In this paper, we propose a robust and efficient scheme for fiducial marker tracking. Firstly, we theoretically prove the upper bound of the transformation deviation of aligning the positions of fiducial markers on two micrographs by affine transformation. Secondly, we design an automatic algorithm based on the Gaussian mixture model to accelerate the procedure of fiducial marker tracking. Thirdly, we propose a divide-and-conquer strategy against lens distortions to ensure the reliability of our scheme. To our knowledge, this is the first attempt that theoretically relates the projection model with the tracking model. The real-world experimental results further support our theoretical bound and demonstrate the effectiveness of our algorithm. This work facilitates the fully automatic tracking for datasets with a massive number of fiducial markers. Availability and implementation The C/C ++ source code that implements the fast fiducial marker tracking is available at https://github.com/icthrm/gmm-marker-tracking. Markerauto 1.6 version or later (also integrated in the AuTom platform at http://ear.ict.ac.cn/) offers a complete implementation for fast alignment, in which fast fiducial marker tracking is available by the ‘-t’ option. Supplementary information Supplementary data are available at Bioinformatics online.

Visualization of 3D high-speed biological events, e.g., signaling of neural circuits, has been the driving force of many emerging 3D imaging techniques, such as random-access microscopy1,2 and light-sheet microscopy (LSM).3,4 The former approach selectively scans the laser beam in the regions of interest to monitor biological events at speeds up to 10-s kHz,1,2 and the latter approach exploits a light sheet to parallelly illuminate and record 2D images at frame rates up to 100-s Hz;3,4 accordingly, clear 3D images can be obtained through the application of custom-designed phase masks3 or synchronization with an axial scanner,4 achieving a volume image rate of ∼30 Hz. Light-field microscopy (LFM)5 is a high-speed 3D imaging technique that captures a volume image via a single exposure from a camera, where the imaging speed is limited by cameras. Briefly, a 3D image in LFM can be restored from a recorded “light-field image” that simultaneously contains the positions of objects, i.e., 2D image, and angles of illumination, i.e., another 2D image, which is termed a “4D” image. Comparing with other fast fluorescent imaging methods, LFM achieves the highest volume imaging rate at the expense of low spatial and angular resolution,5,6 which prevents the broad adoption of LFM in biological studies. To address the issue, both computational and optical methods have been developed. For example, deconvolution algorithms have been applied to LFM7 for imaging neuronal activities at high speed.8–10 Although simple and effective, this method is prone to generating artifacts near the native image plane (NIP) and requires powerful computers for processing dense 3D images. In addition to the deconvolution algorithm, the phase space retrieval method11 has been developed to computationally recover a high-resolution light-field image. Comparing with the deconvolution method, this method shows decreasing fidelity away from the NIP. In terms of optical methods, insertion of phase masks,12 optimization of optical distances,13 adoption of an orthogonal detection arm,14 and selective plane illumination15 have been demonstrated to improve the spatial and angular resolution of LFM. However, the increased complexity in optical design often prevents the easy application of computational methods, e.g., deconvolution algorithms. In this work, we present a new hybrid LFM via selective optical illumination, which reconstructs a high-resolution 4D light-field image from a simultaneously recorded 2D image and low-resolution 4D light-field image. Briefly, multi-layer 2D images within the depth of the focus (DOF) are generated via deconvolution. Next, the multi-layer images are used as constraints for the iterated Fourier transform of the interpolated 4D light-field image to reconstruct a high-resolution 4D light-field image. Accordingly, the hybrid LFM achieves an imaging speed of 100-s volumes per second (vps) with a 3D field of view of 250×250×80 μm3 in the x,y, and z axes, respectively. The measured axial and lateral resolutions, i.e., full width at half maximum (FWHM), are 9.36 and 1.85 μm, respectively.

Motivation Deep learning use for quantitative image analysis is exponentially increasing. However, training accurate, widely deployable deep learning algorithms requires a plethora of annotated (ground truth) data. Image collections must contain not only thousands of images to provide sufficient example objects (i.e. cells), but also contain an adequate degree of image heterogeneity. Results We present a new dataset, EVICAN—Expert visual cell annotation, comprising partially annotated grayscale images of 30 different cell lines from multiple microscopes, contrast mechanisms and magnifications that is readily usable as training data for computer vision applications. With 4600 images and ∼26 000 segmented cells, our collection offers an unparalleled heterogeneous training dataset for cell biology deep learning application development. Availability and implementation The dataset is freely available (https://edmond.mpdl.mpg.de/imeji/collection/l45s16atmi6Aa4sI?q=). Using a Mask R-CNN implementation, we demonstrate automated segmentation of cells and nuclei from brightfield images with a mean average precision of 61.6 % at a Jaccard Index above 0.5.

Motivation Sputum in the trachea is hard to expectorate and detect directly for the patients who are unconscious, especially those in Intensive Care Unit. Medical staff should always check the condition of sputum in the trachea. This is time-consuming and the necessary skills are difficult to acquire. Currently, there are few automatic approaches to serve as alternatives to this manual approach. Results We develop an automatic approach to diagnose the condition of the sputum. Our approach utilizes a system involving a medical device and quantitative analytic methods. In this approach, the time-frequency distribution of respiratory sound signals, determined from the spectrum, is treated as an image. The sputum detection is performed by interpreting the patterns in the image through the procedure of preprocessing and feature extraction. In this study, 272 respiratory sound samples (145 sputum sound and 127 non-sputum sound samples) are collected from 12 patients. We apply the method of leave-one out cross-validation to the 12 patients to assess the performance of our approach. That is, out of the 12 patients, 11 are randomly selected and their sound samples are used to predict the sound samples in the remaining one patient. The results show that our automatic approach can classify the sputum condition at an accuracy rate of 83.5%. Availability and implementation The matlab codes and examples of datasets explored in this work are available at Bioinformatics online. Supplementary information Supplementary data are available at Bioinformatics online.

Motivation Progress in 3D electron microscopy (EM) imaging has greatly facilitated neuroscience research in high-throughput data acquisition. Correspondingly, high-throughput automated image analysis methods are necessary to work on par with the speed of data being produced. One such example is the need for automated EM image segmentation for neurite reconstruction. However, the efficiency and reliability of current methods are still lagging far behind human performance. Results Here, we propose DeepEM3D, a deep learning method for segmenting 3D anisotropic brain electron microscopy images. In this method, the deep learning model can efficiently build feature representation and incorporate sufficient multi-scale contextual information. We propose employing a combination of novel boundary map generation methods with optimized model ensembles to address the inherent challenges of segmenting anisotropic images. We evaluated our method by participating in the 3D segmentation of neurites in EM images (SNEMI3D) challenge. Our submission is ranked #1 on the current leaderboard as of Oct 15, 2016. More importantly, our result was very close to human-level performance in terms of the challenge evaluation metric: namely, a Rand error of 0.06015 versus the human value of 0.05998. Availability and Implementation The code is available at https://github.com/divelab/deepem3d/ Supplementary information Supplementary data are available at Bioinformatics online.

Motivation Current spectral unmixing methods for multiplex fluorescence microscopy have a limited ability to cope with high spectral overlap as they only analyze spectral information over individual pixels. Here, we present adaptive Morphologically Constrained Spectral Unmixing (MCSU) algorithms that overcome this limitation by exploiting morphological differences between sub-cellular structures, and their local spatial context. Results The proposed method was effective at improving spectral unmixing performance by exploiting: (i) a set of dictionary-based models for object morphologies learned from the image data; and (ii) models of spatial context learned from the image data using a total variation algorithm. The method was evaluated on multi-spectral images of multiplex-labeled pancreatic ductal adenocarcinoma (PDAC) tissue samples. The former constraint ensures that neighbouring pixels correspond to morphologically similar structures, and the latter constraint ensures that neighbouring pixels have similar spectral signatures. The average Mean Squared Error (MSE) and Signal Reconstruction Error (SRE) ratio of the proposed method was 39.6% less and 8% more, respectively, compared to the best of all other algorithms that do not exploit these spatial constraints. Availability and Implementation Open source software (MATLAB). Supplementary information Supplementary data are available at Bioinformatics online.

Blood flow is a parameter used to diagnose vascular diseases based on flow speed, blood pressure, and vessel size. Different techniques have been developed to estimate the relative blood flow speed and to improve the visualization of deep blood vessels; one such technique is laser speckle contrast imaging (LSCI). LSCI images contain a high level of noise mainly when deep blood vessels are imaged. To improve their visualization, several approaches for contrast computation have been developed. However, there is a compromise between noise attenuation and temporal resolution. On the one hand, spatial approaches have low spatial resolution, high temporal resolution, and significant noise attenuation, while temporal approaches have the opposite. A recent approach combines a temporal base with a directional process that allows improving the visualization of blood vessels. Nevertheless, it still contains a high level of noise and requires a high number of raw frames for its base. We propose, a space-directional approach focused on improving noise attenuation and temporal resolution for contrast computation. The results of reference approaches and the proposed one are compared quantitatively. Moreover, it is shown that the visualization of blood vessels in LSCI images can be improved by a general morphological process when the noise level is reduced.

Objective: To analyse the effect of different image reconstruction techniques on image quality and dual energy CT (DECT) imaging metrics.Methods: A software platform for pre-clinical cone beam CT X-ray image reconstruction was built using the open-source reconstruction toolkit. Pre-processed projections were reconstructed with filtered back-projection and iterative algorithms, namely Feldkamp, Davis, and Kress (FDK), Iterative FDK, simultaneous algebraic reconstruction technique (SART), simultaneous iterative reconstruction technique and conjugate gradient. Imaging metrics were quantitatively assessed, using a quality assurance phantom, and DECT analysis was performed to determine the influence of each reconstruction technique on the relative electron density (rho(e)) and effective atomic number (Z(eff)) values.Results: Iterative reconstruction had favourable results for the DECT analysis: a significantly smaller spread for each material in the rho(e)-Z(eff) space and lower Z(eff) and rho(e) residuals (on average 24 and 25% lower, respectively). In terms of image quality assurance, the techniques FDK, Iterative FDK and SART provided acceptable results. The three reconstruction methods showed similar geometric accuracy, uniformity and CT number results. The technique SART had a contrast-to-noise ratio up to 76% higher for solid water and twice as high for Teflon, but resolution was up to 28% lower when compared to the other two techniques.Conclusions: Advanced image reconstruction can be beneficial, but the benefit is small, and calculation times may be unacceptable with current technology. The use of targeted and downscaled reconstruction grids, larger, yet practicable, pixel sizes and GPU are recommended.Advances in knowledge: An iterative CBCT reconstruction platform was build using RTK.

Background Multimodal wearable technologies have brought forward wide possibilities in human activity recognition, and more specifically personalized monitoring of eating habits. The emerging challenge now is the selection of most discriminative information from high-dimensional data collected from multiple sources. The available fusion algorithms with their complex structure are poorly adopted to the computationally constrained environment which requires integrating information directly at the source. As a result, more simple low-level fusion methods are needed. Objective In the absence of a data combining process, the cost of directly applying high-dimensional raw data to a deep classifier would be computationally expensive with regard to the response time, energy consumption, and memory requirement. Taking this into account, we aimed to develop a data fusion technique in a computationally efficient way to achieve a more comprehensive insight of human activity dynamics in a lower dimension. The major objective was considering statistical dependency of multisensory data and exploring intermodality correlation patterns for different activities. Methods In this technique, the information in time (regardless of the number of sources) is transformed into a 2D space that facilitates classification of eating episodes from others. This is based on a hypothesis that data captured by various sensors are statistically associated with each other and the covariance matrix of all these signals has a unique distribution correlated with each activity which can be encoded on a contour representation. These representations are then used as input of a deep model to learn specific patterns associated with specific activity. Results In order to show the generalizability of the proposed fusion algorithm, 2 different scenarios were taken into account. These scenarios were different in terms of temporal segment size, type of activity, wearable device, subjects, and deep learning architecture. The first scenario used a data set in which a single participant performed a limited number of activities while wearing the Empatica E4 wristband. In the second scenario, a data set related to the activities of daily living was used where 10 different participants wore inertial measurement units while performing a more complex set of activities. The precision metric obtained from leave-one-subject-out cross-validation for the second scenario reached 0.803. The impact of missing data on performance degradation was also evaluated. Conclusions To conclude, the proposed fusion technique provides the possibility of embedding joint variability information over different modalities in just a single 2D representation which results in obtaining a more global view of different aspects of daily human activities at hand, and yet preserving the desired performance level in activity recognition.

We study a problem scenario of super-resolution (SR) algorithms in the context of whole slide imaging (WSI), a popular imaging modality in digital pathology. Instead of just one pair of high- and low-resolution images, which is typically the setup in which SR algorithms are designed, we are given multiple intermediate resolutions of the same image as well. The question remains how to best utilize such data to make the transformation learning problem inherent to SR more tractable and address the unique challenges that arises in this biomedical application. We propose a recurrent convolutional neural network model, to generate SR images from such multi-resolution WSI datasets. Specifically, we show that having such intermediate resolutions is highly effective in making the learning problem easily trainable and address large resolution difference in the low and high-resolution images common in WSI, even without the availability of a large size training data. Experimental results show state-of-the-art performance on three WSI histopathology cancer datasets, across a number of metrics.

Objective: To compare image quality and breast density of two reconstruction methods, the widely-used filtered-back projection (FBP) reconstruction and the iterative heuristic Bayesian inference reconstruction (Bayesian inference reconstruction plus the method of total variation applied, HBI). Methods: Thirty-two clinical DBT data sets with malignant and benign findings, n = 27 and 17, respectively, were reconstructed using FBP and HBI. Three experienced radiologists evaluated the images independently using a 5-point visual grading scale and classified breast density according to the American College of Radiology Breast Imaging-Reporting And Data System Atlas, fifth edition. Image quality metrics included lesion conspicuity, clarity of lesion borders and spicules, noise level, artifacts surrounding the lesion, visibility of parenchyma and breast density. Results: For masses, the image quality of HBI reconstructions was superior to that of FBP in terms of conspicuity,clarity of lesion borders and spicules (p < 0.01). HBI and FBP were not significantly different in calcification conspicuity. Overall, HBI reduced noise and supressed artifacts surrounding the lesions better (p < 0.01). The visibility of fibroglandular parenchyma increased using the HBI method (p < 0.01). On average, five cases per radiologist were downgraded from BI-RADS breast density category C/D to A/B. Conclusion: HBI significantly improves lesion visibility compared to FBP. HBI-visibility of breast parenchyma increased, leading to a lower breast density rating. Applying the HBIR algorithm should improve the diagnostic performance of DBT and decrease the need for additional imaging in patients with dense breasts. Advances in knowledge: Iterative heuristic Bayesian inference (HBI) image reconstruction substantially improves the image quality of breast tomosynthesis leading to a better visibility of breast carcinomas and reduction of the perceived breast density compared to the widely-used filtered-back projection (FPB) reconstruction. Applying HBI should improve the accuracy of breast tomosynthesis and reduce the number of unnecessary breast biopsies. It may also reduce the radiation dose for the patients, which is especially important in the screening context.

Motivation: There is a need for effective automated methods for profiling dynamic cell–cell interactions with single-cell resolution from high-throughput time-lapse imaging data, especially, the interactions between immune effector cells and tumor cells in adoptive immunotherapy. Results: Fluorescently labeled human T cells, natural killer cells (NK), and various target cells (NALM6, K562, EL4) were co-incubated on polydimethylsiloxane arrays of sub-nanoliter wells (nanowells), and imaged using multi-channel time-lapse microscopy. The proposed cell segmentation and tracking algorithms account for cell variability and exploit the nanowell confinement property to increase the yield of correctly analyzed nanowells from 45% (existing algorithms) to 98% for wells containing one effector and a single target, enabling automated quantification of cell locations, morphologies, movements, interactions, and deaths without the need for manual proofreading. Automated analysis of recordings from 12 different experiments demonstrated automated nanowell delineation accuracy >99%, automated cell segmentation accuracy >95%, and automated cell tracking accuracy of 90%, with default parameters, despite variations in illumination, staining, imaging noise, cell morphology, and cell clustering. An example analysis revealed that NK cells efficiently discriminate between live and dead targets by altering the duration of conjugation. The data also demonstrated that cytotoxic cells display higher motility than non-killers, both before and during contact. Contact: broysam@central.uh.edu or nvaradar@central.uh.edu Supplementary information: Supplementary data are available at Bioinformatics online.

We propose a three-dimensional (3-D) registration method to correct motion artifacts and construct the volume structure for angiographic and structural optical coherence tomography (OCT). This algorithm is particularly suitable for the nonorthogonal wide-field OCT scan acquired by a ultrahigh-speed swept-source system ( > 200 ?? kHz A-scan rate). First, the transverse motion artifacts are corrected by the between-frame registration based on en face OCT angiography (OCTA). After A-scan transverse translation between B-frames, the axial motions are corrected based on the rebuilt boundary of inner limiting membrane. Finally, a within-frame registration is performed for local optimization based on cross-sectional OCTA. We evaluated this algorithm on retinal volumes of six normal subjects. The results showed significantly improved retinal smoothness in 3-D-registered structural OCT and image contrast on en face OCTA.

In conventional retinal region detection methods for optical coherence tomography (OCT) images, many parameters need to be set manually, which is often detrimental to their generalizability. We present a scheme to detect retinal regions based on fully convolutional networks (FCN) for automatic diagnosis of abnormal maculae in OCT images. The FCN model is trained on 900 labeled age-related macular degeneration (AMD), diabetic macular edema (DME) and normal (NOR) OCT images. Its segmentation accuracy is validated and its effectiveness in recognizing abnormal maculae in OCT images is tested and compared with traditional methods, by using the spatial pyramid matching based on sparse coding (ScSPM) classifier and Inception V3 classifier on two datasets: Duke dataset and our clinic dataset. In our clinic dataset, we randomly selected half of the B-scans of each class (300 AMD, 300 DME, and 300 NOR) for training classifier and the rest (300 AMD, 300 DME, and 300 NOR) for testing with 10 repetitions. Average accuracy, sensitivity, and specificity of 98.69%, 98.03%, and 99.01% are obtained by using ScSPM classifier, and those of 99.69%, 99.53%, and 99.77% are obtained by using Inception V3 classifier. These two classification algorithms achieve 100% classification accuracy when directly applied to Duke dataset, where all the 45 OCT volumes are used as test set. Finally, FCN model with or without flattening and cropping and its influence on classification performance are discussed.

Optical coherence tomography angiography (OCTA) provides in-vivo images of microvascular perfusion in high resolution. For its application to basic and clinical research, an automatic and robust quantification of the capillary architecture is mandatory. Only this makes it possible to reliably analyze large amounts of image data, to establish biomarkers, and to monitor disease developments. However, due to its optical properties, OCTA images of skin often suffer from a poor signal-to-noise ratio and contain imaging artifacts. Previous work on automatic vessel segmentation in OCTA mostly focuses on retinal and cerebral vasculature. Its applicability to skin and, furthermore, its robustness against imaging artifacts had not been systematically evaluated. We propose a segmentation method that improves the quality of vascular quantification in OCTA images even if corrupted by imaging artifacts. Both the combination of image processing methods and the choice of their parameters are systematically optimized to match the manual labeling of an expert for OCTA images of skin. The efficacy of this optimization-based vessel segmentation is further demonstrated on sample images as well as by a reduced error of derived quantitative vascular network characteristics.

We developed a novel method of producing accurate range images of the velopharynx using a three-dimensional (3D) endoscope to obtain detailed measurements of velopharyngeal movements. The purpose of the present study was to determine the relationship between the distance from the endoscope to an object, elucidate the measurement accuracy along the temporal axes, and determine the degree of blurring when using a jig to fix the endoscope.An endoscopic measuring system was developed in which a pattern projection system was incorporated into a commercially available 3D endoscope. After correcting the distortion of the camera images, range images were produced using pattern projection to achieve stereo matching. Graph paper was used to measure the appropriate distance from the camera to an object, the mesial buccal cusp of the right maxillary first molar was measured to clarify the range image stability, and an electric actuator was used to evaluate the measurement accuracy along the temporal axes.The measurement error was substantial when the distance from the camera to the subject was6.5 cm. The standard error of the 3D coordinate value produced from 30 frames was within 0.1 mm (range, 0.01-0.08 mm). The measurement error of the temporal axes was 9.16% in the horizontal direction and 9.27% in the vertical direction.The optimal distance from the camera to an object is6.5 cm. The present endoscopic measuring system can provide stable range images of the velopharynx when using an appropriate fixation method and enables quantitative analysis of velopharyngeal movements.

Motivation: Knowledge of the subcellular location of a protein is crucial for understanding its functions. The subcellular pattern of a protein is typically represented as the set of cellular components in which it is located, and an important task is to determine this set from microscope images. In this article, we address this classification problem using confocal immunofluorescence images from the Human Protein Atlas (HPA) project. The HPA contains images of cells stained for many proteins; each is also stained for three reference components, but there are many other components that are invisible. Given one such cell, the task is to classify the pattern type of the stained protein. We first randomly select local image regions within the cells, and then extract various carefully designed features from these regions. This region-based approach enables us to explicitly study the relationship between proteins and different cell components, as well as the interactions between these components. To achieve these two goals, we propose two discriminative models that extend logistic regression with structured latent variables. The first model allows the same protein pattern class to be expressed differently according to the underlying components in different regions. The second model further captures the spatial dependencies between the components within the same cell so that we can better infer these components. To learn these models, we propose a fast approximate algorithm for inference, and then use gradient-based methods to maximize the data likelihood. Results: In the experiments, we show that the proposed models help improve the classification accuracies on synthetic data and real cellular images. The best overall accuracy we report in this article for classifying 942 proteins into 13 classes of patterns is about 84.6%, which to our knowledge is the best so far. In addition, the dependencies learned are consistent with prior knowledge of cell organization. Availability: http://murphylab.web.cmu.edu/software/. Contact: Jeff.Schneider@cs.cmu.edu, murphy@cmu.edu

We introduce a metric in graph search and demonstrate its application for segmenting retinal optical coherence tomography (OCT) images of macular pathology. Our proposed “adjusted mean arc length” (AMAL) metric is an adaptation of the lowest mean arc length search technique for automated OCT segmentation. We compare this method to Dijkstra’s shortest path algorithm, which we utilized previously in our popular graph theory and dynamic programming segmentation technique. As an illustrative example, we show that AMAL-based length-adaptive segmentation outperforms the shortest path in delineating the retina/vitreous boundary of patients with full-thickness macular holes when compared with expert manual grading.

Motivation: The medical imaging and image processing techniques, ranging from microscopic to macroscopic, has become one of the main components of diagnostic procedures to assist dermatologists in their medical decision-making processes. Computer-aided segmentation and border detection on dermoscopic images is one of the core components of diagnostic procedures and therapeutic interventions for skin cancer. Automated assessment tools for dermoscopic images have become an important research field mainly because of inter- and intra-observer variations in human interpretations. In this study, a novel approach—graph spanner—for automatic border detection in dermoscopic images is proposed. In this approach, a proximity graph representation of dermoscopic images in order to detect regions and borders in skin lesion is presented. Results: Graph spanner approach is examined on a set of 100 dermoscopic images whose manually drawn borders by a dermatologist are used as the ground truth. Error rates, false positives and false negatives along with true positives and true negatives are quantified by digitally comparing results with manually determined borders from a dermatologist. The results show that the highest precision and recall rates obtained to determine lesion boundaries are 100%. However, accuracy of assessment averages out at 97.72% and borders errors' mean is 2.28% for whole dataset. Contact: skockara@uca.edu

Functional brain imaging techniques require accurate co-registration to anatomical images to precisely identify the areas being activated. Many of them, including diffuse optical imaging, rely on scalp-placed recording sensors. Fiducial alignment is an effective and rapid method for co-registering scalp sensors onto anatomy, but is quite sensitive to placement errors. Surface Euclidean distance minimization using the Levenberq-Marquart algorithm (LMA) has been shown to be very accurate when based on good initial guesses, such as precise fiducial alignment, but its accuracy drops substantially with fiducial placement errors. Here we compared fiducial and LMA co-registration methods to a new procedure, the iterative closest point-to-plane (ICP2P) method, using simulated and real data. An advantage of ICP2P is that it eliminates the need to identify fiducials and is, therefore, entirely automatic. We show that, typically, ICP2P is as accurate as fiducial-based LMA, but is less sensitive to initial placement errors. However, ICP2P is more sensitive to spatially correlated noise in the description of the head surface. Hence, the best technique for co-registration depends on the type of data available to describe the scalp and the surface defined by the recording sensors. Under optimal conditions, co-registration error using surface-fitting procedures can be reduced to ~ 3 mm.

We present a prototype software system with sufficient capacity and speed to estimate radiation exposures in a mass casualty event by counting dicentric chromosomes (DCs) in metaphase cells from many individuals. Top-ranked metaphase cell images are segmented by classifying and defining chromosomes with an active contour gradient vector field (GVF) and by determining centromere locations along the centreline. The centreline is extracted by discrete curve evolution (DCE) skeleton branch pruning and curve interpolation. Centromere detection minimises the global width and DAPI-staining intensity profiles along the centreline. A second centromere is identified by reapplying this procedure after masking the first. Dicentrics can be identified from features that capture width and intensity profile characteristics as well as local shape features of the object contour at candidate pixel locations. The correct location of the centromere is also refined in chromosomes with sister chromatid separation. The overall algorithm has both high sensitivity (85 %) and specificity (94 %). Results are independent of the shape and structure of chromosomes in different cells, or the laboratory preparation protocol followed. The prototype software was recoded in C++/OpenCV; image processing was accelerated by data and task parallelisation with Message Passaging Interface and Intel Threading Building Blocks and an asynchronous non-blocking I/O strategy. Relative to a serial process, metaphase ranking, GVF and DCE are, respectively, 100 and 300-fold faster on an 8-core desktop and 64-core cluster computers. The software was then ported to a 1024-core supercomputer, which processed 200 metaphase images each from 1025 specimens in 1.4 h.

Summarization: Automated algorithms are developed in this paper based on digital signal processing to correlate electron-optical data from skin collagen fibrils with known sequence data. In particular, they detect alterations on the ratio of major collagen types in skin upon age. Presented on: Micron

It is necessary to extract target specimens from bioholographic images for high-level analysis such as object identification, recognition, and tracking with the advent of application of digital holographic microscopy to transparent or semi-transparent biological specimens. We present an interactive graph cuts approach to segment the needed target specimens in the reconstructed bioholographic images. This method combines both regional and boundary information and is robust to extract targets with weak boundaries. Moreover, this technique can achieve globally optimal results while minimizing an energy function. We provide a convenient user interface, which can easily differentiate the foreground/background for various types of holographic images, as well as a dynamically modified coefficient, which specifies the importance of the regional and boundary information. The extracted results from our scheme have been compared with those from an advanced level-set-based segmentation method using an unbiased comparison algorithm. Experimental results show that this interactive graph cut technique can not only extract different kinds of target specimens in bioholographic images, but also yield good results when there are multiple similar objects in the holographic image or when the object boundaries are very weak.

We demonstrate the use of task-based image-quality metrics to compare various photoacoustic image-reconstruction algorithms, including a method based on the pseudoinverse of the system matrix, simple backprojection, filtered backprojection, and a method based on the Fourier transform. We use a three-dimensional forward model with a linear transducer array to simulate a photoacoustic imaging system. The reconstructed images correspond with two-dimensional slices of the object and are 128×128 pixels. In order to compare the algorithms, we use channelized Hotelling observers that predict the detection ability of human observers. We use two sets of channels: constant Q and difference of Gaussian spatial frequency channels. We look at three tasks, identification of a point source in a uniform background, identification of a 0.5-mm cube in a uniform background, and identification of a point source in a lumpy background. For the lumpy background task, which is the most realistic of the tasks, the method based on the pseudoinverse performs best according to both sets of channels.

We achieved fast and automatic retinal vessel segmentation by employing the active contours method in Doppler spectral-domain optical coherence tomography (SD-OCT). In a typical OCT B-scan image, we first extracted the phase variations between adjacent A-lines and removed bulk motion. Then we set the initial contour as the boundary of the whole image and iterated until all of the segmented vessel contours became stabilized. Using a typical office computer, the whole segmentation took no more than 50 s, making real-time retinal vessel segmentation possible. We tested the active contours method segmentation in both controlled phantom and in vivo rodent eye images.

Spatial frequency domain imaging (SFDI) is a noncontact and wide-field optical imaging technology currently being used to study the optical properties and chromophore concentrations of in vivo skin including skin lesions of various types. Part of the challenge of developing a clinically deployable SFDI system is related to the development of effective motion compensation strategies, which in turn, is critical for recording high fidelity optical properties. Here we present a two-part strategy for SFDI motion correction. After verifying the effectiveness of the motion correction algorithm on tissue-simulating phantoms, a set of skin-imaging data was collected in order to test the performance of the correction technique under real clinical conditions. Optical properties were obtained with and without the use of the motion correction technique. The results indicate that the algorithm presented here can be used to render optical properties in moving skin surfaces with fidelities within 1.5% of an ideal stationary case and with up to 92.63% less variance. Systematic characterization of the impact of motion variables on clinical SFDI measurements reveals that until SFDI instrumentation is developed to the point of instantaneous imaging, motion compensation is necessary for the accurate localization and quantification of heterogeneities in a clinical setting. © 2011 Society of Photo-Optical Instrumentation Engineers (SPIE).

The purpose of this study was to develop, implement and evaluate a dedicated metal artefact reduction (MAR) method for flat-detector CT (FDCT). The algorithm uses the multidimensional raw data space to calculate surrogate attenuation values for the original metal traces in the raw data domain. The metal traces are detected automatically by a three-dimensional, threshold-based segmentation algorithm in an initial reconstructed image volume, based on twofold histogram information for calculating appropriate metal thresholds. These thresholds are combined with constrained morphological operations in the projection domain. A subsequent reconstruction of the modified raw data yields an artefact-reduced image volume that is further processed by a combining procedure that reinserts the missing metal information. For image quality assessment, measurements on semi-anthropomorphic phantoms containing metallic inserts were evaluated in terms of CT value accuracy, image noise and spatial resolution before and after correction. Measurements of the same phantoms without prostheses were used as ground truth for comparison. Cadaver measurements were performed on complex and realistic cases and to determine the influences of our correction method on the tissue surrounding the prostheses. The results showed a significant reduction of metal-induced streak artefacts (CT value differences were reduced to below 22 HU and image noise reduction of up to 200%). The cadaver measurements showed excellent results for imaging areas close to the implant and exceptional artefact suppression in these areas. Furthermore, measurements in the knee and spine regions confirmed the superiority of our method to standard one-dimensional, linear interpolation.

Motivation: Proteins exhibit complex subcellular distributions, which may include localizing in more than one organelle and varying in location depending on the cell physiology. Estimating the amount of protein distributed in each subcellular location is essential for quantitative understanding and modeling of protein dynamics and how they affect cell behaviors. We have previously described automated methods using fluorescent microscope images to determine the fractions of protein fluorescence in various subcellular locations when the basic locations in which a protein can be present are known. As this set of basic locations may be unknown (especially for studies on a proteome-wide scale), we here describe unsupervised methods to identify the fundamental patterns from images of mixed patterns and estimate the fractional composition of them. Methods: We developed two approaches to the problem, both based on identifying types of objects present in images and representing patterns by frequencies of those object types. One is a basis pursuit method (which is based on a linear mixture model), and the other is based on latent Dirichlet allocation (LDA). For testing both approaches, we used images previously acquired for testing supervised unmixing methods. These images were of cells labeled with various combinations of two organelle-specific probes that had the same fluorescent properties to simulate mixed patterns of subcellular location. Results: We achieved 0.80 and 0.91 correlation between estimated and underlying fractions of the two probes (fundamental patterns) with basis pursuit and LDA approaches, respectively, indicating that our methods can unmix the complex subcellular distribution with reasonably high accuracy. Availability: http://murphylab.web.cmu.edu/software Contact: murphy@cmu.edu

A system for scoring images based on the likelihood of containing crystalline material is described. A simulation carried out on a real set of crystallization images demonstrates the utility of such a system in high-throughput environments by substantially reducing the manual workload necessary to detect crystals for X-ray screening., The ability of computers to learn from and annotate large databases of crystallization-trial images provides not only the ability to reduce the workload of crystallization studies, but also an opportunity to annotate crystallization trials as part of a framework for improving screening methods. Here, a system is presented that scores sets of images based on the likelihood of containing crystalline material as perceived by a machine-learning algorithm. The system can be incorporated into existing crystallization-analysis pipelines, whereby specialists examine images as they normally would with the exception that the images appear in rank order according to a simple real-valued score. Promising results are shown for 319 112 images associated with 150 structures solved by the Joint Center for Structural Genomics pipeline during the 2006–2007 year. Overall, the algorithm achieves a mean receiver opera­ting characteristic score of 0.919 and a 78% reduction in human effort per set when considering an absolute score cutoff for screening images, while incurring a loss of five out of 150 structures.

Most studies evaluating the potential of optical coherence tomography (OCT) for the diagnosis of oral cancer are based on visual assessment of OCT B-scans by trained experts. Human interpretation of the large pool of data acquired by modern high-speed OCT systems, however, can be cumbersome and extremely time consuming. Development of image analysis methods for automated and quantitative OCT image analysis could therefore facilitate the evaluation of such a large volume of data. We report automated algorithms for quantifying structural features that are associated with the malignant transformation of the oral epithelium based on image processing of OCT data. The features extracted from the OCT images were used to design a statistical classification model to perform the automated tissue diagnosis. The sensitivity and specificity of distinguishing malignant lesions from benign lesions were found to be 90.2% and 76.3%, respectively. The results of the study demonstrate the feasibility of using quantitative image analysis algorithms for extracting morphological features from OCT images to perform the automated diagnosis of oral malignancies in a hamster cheek pouch model.

Multimodal approaches that combine near-infrared (NIR) and conventional imaging modalities have been shown to improve optical parameter estimation dramatically and thus represent a prevailing trend in NIR imaging. These approaches typically involve applying anatomical templates from magnetic resonance imaging/computed tomography/ultrasound images to guide the recovery of optical parameters. However, merging these data sets using current technology requires multiple software packages, substantial expertise, significant time-commitment, and often results in unacceptably poor mesh quality for optical image reconstruction, a reality that represents a significant roadblock for translational research of multimodal NIR imaging. This work addresses these challenges directly by introducing automated digital imaging and communications in medicine image stack segmentation and a new one-click three-dimensional mesh generator optimized for multimodal NIR imaging, and combining these capabilities into a single software package (available for free download) with a streamlined workflow. Image processing time and mesh quality benchmarks were examined for four common multimodal NIR use-cases (breast, brain, pancreas, and small animal) and were compared to a commercial image processing package. Applying these tools resulted in a fivefold decrease in image processing time and 62% improvement in minimum mesh quality, in the absence of extra mesh postprocessing. These capabilities represent a significant step toward enabling translational multimodal NIR research for both expert and nonexpert users in an open-source platform.