Search

Background and aims Although plaque characterization by intravascular ultrasound (IVUS) is important for risk stratification, frame-by-frame analysis of a whole vascular segment is time-consuming. The aim was to develop IVUS-based algorithms for classifying attenuation and calcified plaques. Methods IVUS image sets of 598 coronary arteries from 598 patients were randomized into training and test sets with 5:1 ratio. Each IVUS frame at a 0.4-mm interval was circumferentially labeled as one of three classes: attenuated plaque, calcified plaque, or plaque without attenuation or calcification. The model was trained on multi-class classification with 5-fold cross validation. By converting from Cartesian to polar coordinate images, the class corresponding to each array from 0 to 360° was plotted. Results At the angle-level, Dice similarity coefficients for identifying calcification vs. attenuation vs. none by using ensemble model were 0.79, 0.74 and 0.99, respectively. Also, the maximal accuracy was 98% to classify those groups in the test set. At the frame-level, the model identified the presence of attenuation with 80% sensitivity, 96% specificity, and 93% overall accuracy, and the presence of calcium with 86% sensitivity, 97% specificity, and 96% overall accuracy. In the per-vessel analysis, the attenuation and calcification burden index closely correlated with human measurements (r = 0.89 and r = 0.95, respectively), as did the maximal attenuation and calcification burden index over 4 mm (r = 0.82 and r = 0.91, respectively). The inference times were 0.05 s per frame and 7.8 s per vessel. Conclusions Our deep learning algorithms for plaque characterization may assist clinicians in recognizing high-risk coronary lesions.

Objectives The aim of this study was to develop pre-procedural intravascular ultrasound (IVUS)–based models for predicting the occurrence of stent underexpansion. Background Although post-stenting IVUS has been used to optimize percutaneous coronary intervention, there are no pre-procedural guidelines to estimate the degree of stent expansion and provide preemptive management before stent deployment. Methods A total of 618 coronary lesions in 618 patients undergoing percutaneous coronary intervention were randomized into training and test sets in a 5:1 ratio. Following the coregistration of pre- and post-stenting IVUS images, the pre-procedural images and clinical information (stent diameter, length, and inflation pressure; balloon diameter; and maximal balloon pressure) were used to develop a regression model using a convolutional neural network to predict post-stenting stent area. To separate the frames with from those without the occurrence of underexpansion (stent area Results Overall, the frequency of stent underexpansion was 15% (5,209 of 34,736 frames). At the frame level, stent areas predicted by the pre-procedural IVUS-based regression model significantly correlated with those measured on post-stenting IVUS (r = 0.802). To predict stent underexpansion, maximal accuracy of 94% (area under the curve = 0.94) was achieved when the convolutional neural network– and mask image–derived features were used for the classification model. At the lesion level, there were significant correlations between predicted and measured minimal stent area (r = 0.832) and between predicted and measured total stent volume (r = 0.958). Conclusions Deep-learning algorithms accurately predicted incomplete stent expansion. A data-driven approach may assist clinicians in making treatment decisions to avoid stent underexpansion as a preventable cause of stent failure.

Optical diffraction tomography measures the three-dimensional refractive index map of a specimen and visualizes biochemical phenomena at the nanoscale in a non-destructive manner. One major drawback of optical diffraction tomography is poor axial resolution due to limited access to the three-dimensional optical transfer function. This missing cone problem has been addressed through regularization algorithms that use a priori information, such as non-negativity and sample smoothness. However, the iterative nature of these algorithms and their parameter dependency make real-time visualization impossible. In this article, we propose and experimentally demonstrate a deep neural network, which we term DeepRegularizer, that rapidly improves the resolution of a three-dimensional refractive index map. Trained with pairs of datasets (a raw refractive index tomogram and a resolution-enhanced refractive index tomogram via the iterative total variation algorithm), the three-dimensional U-net-based convolutional neural network learns a transformation between the two tomogram domains. The feasibility and generalizability of our network are demonstrated using bacterial cells and a human leukaemic cell line, and by validating the model across different samples. DeepRegularizer offers more than an order of magnitude faster regularization performance compared to the conventional iterative method. We envision that the proposed data-driven approach can bypass the high time complexity of various image reconstructions in other imaging modalities.

AIMS The aim of this study was to develop a deep learning model for classifying frames with versus without optical coherence tomography (OCT)-derived thin-cap fibroatheroma (TCFA). METHODS AND RESULTS A total of 602 coronary lesions from 602 angina patients were randomised into training and test sets in a 4:1 ratio. A DenseNet model was developed to classify OCT frames with or without OCT-derived TCFA. Gradient-weighted class activation mapping was used to visualise the area of attention. In the training sample (35,678 frames of 480 lesions), the model with fivefold cross-validation had an overall accuracy of 91.6±1.7%, sensitivity of 88.7±3.4%, and specificity of 91.8±2.0% (averaged AUC=0.96±0.01) in predicting the presence of TCFA. In the test samples (9,722 frames of 122 lesions), the overall accuracy at the frame level was 92.8% within the lesion (AUC=0.96) and 91.3% in the entire OCT pullback. The correlation between the %TCFA burden per vessel predicted by the model compared with that identified by experts was significant (r=0.87, p

The healthcare industry is in dire need of rapid microbial identification techniques for treating microbial infections. Microbial infections are a major healthcare issue worldwide, as these widespread diseases often develop into deadly symptoms. While studies have shown that an early appropriate antibiotic treatment significantly reduces the mortality of an infection, this effective treatment is difficult to practice. The main obstacle to early appropriate antibiotic treatments is the long turnaround time of the routine microbial identification, which includes time-consuming sample growth. Here, we propose a microscopy-based framework that identifies the pathogen from single to few cells. Our framework obtains and exploits the morphology of the limited sample by incorporating three-dimensional quantitative phase imaging and an artificial neural network. We demonstrate the identification of 19 bacterial species that cause bloodstream infections, achieving an accuracy of 82.5% from an individual bacterial cell or cluster. This performance, comparable to that of the gold standard mass spectroscopy under a sufficient amount of sample, underpins the effectiveness of our framework in clinical applications. Furthermore, our accuracy increases with multiple measurements, reaching 99.9% with seven different measurements of cells or clusters. We believe that our framework can serve as a beneficial advisory tool for clinicians during the initial treatment of infections.

Visualisations and analyses of cellular and subcellular organelles in biological cells is crucial for the study of cell biology. However, existing imaging methods require the use of exogenous labelling agents, which prevents the long-time assessments of live cells in their native states. Here we propose and experimentally demonstrate three-dimensional segmentation of subcellular organelles in unlabelled live cells, exploiting a 3D U-Net-based architecture. We present the high-precision three-dimensional segmentation of cell membrane, nucleus membrane, nucleoli, and lipid droplets of various cell types. Time-lapse analyses of dynamics of activated immune cells are also analysed using label-free segmentation. O_FIG O_LINKSMALLFIG WIDTH=200 HEIGHT=113 SRC="FIGDIR/small/445351v1_ufig1.gif" ALT="Figure 1"> View larger version (34K): org.highwire.dtl.DTLVardef@b92093org.highwire.dtl.DTLVardef@9caa8org.highwire.dtl.DTLVardef@d81408org.highwire.dtl.DTLVardef@b3c87_HPS_FORMAT_FIGEXP M_FIG C_FIG

We present a deep learning approach for the rapid resolution enhancement of optical diffraction tomography. Once our three-dimensional U-net-based convolutional neural network learns an image translation between raw tomograms and total-variation-regularized tomograms, the trained network can fill in the missing cone of a measured refractive index tomogram and improve its resolution within seconds. We demonstrate the feasibility and generalizability of our approach on various biological samples, including bacteria, WBC, and NIH3T3.

Objective and Impact Statement . We propose a rapid and accurate blood cell identification method exploiting deep learning and label-free refractive index (RI) tomography. Our computational approach that fully utilizes tomographic information of bone marrow (BM) white blood cell (WBC) enables us to not only classify the blood cells with deep learning but also quantitatively study their morphological and biochemical properties for hematology research. Introduction . Conventional methods for examining blood cells, such as blood smear analysis by medical professionals and fluorescence-activated cell sorting, require significant time, costs, and domain knowledge that could affect test results. While label-free imaging techniques that use a specimen’s intrinsic contrast (e.g., multiphoton and Raman microscopy) have been used to characterize blood cells, their imaging procedures and instrumentations are relatively time-consuming and complex. Methods . The RI tomograms of the BM WBCs are acquired via Mach-Zehnder interferometer-based tomographic microscope and classified by a 3D convolutional neural network. We test our deep learning classifier for the four types of bone marrow WBC collected from healthy donors ( n = 10 ): monocyte, myelocyte, B lymphocyte, and T lymphocyte. The quantitative parameters of WBC are directly obtained from the tomograms. Results . Our results show >99% accuracy for the binary classification of myeloids and lymphoids and >96% accuracy for the four-type classification of B and T lymphocytes, monocyte, and myelocytes. The feature learning capability of our approach is visualized via an unsupervised dimension reduction technique. Conclusion . We envision that the proposed cell classification framework can be easily integrated into existing blood cell investigation workflows, providing cost-effective and rapid diagnosis for hematologic malignancy.

The immunological synapse (IS) is a cell-cell junction between a T cell and a professional antigen-presenting cell. Since the IS formation is a critical step for the initiation of an antigen-specific immune response, various live-cell imaging techniques, most of which rely on fluorescence microscopy, have been used to study the dynamics of IS. However, the inherent limitations associated with the fluorescence-based imaging, such as photo-bleaching and photo-toxicity, prevent the long-term assessment of dynamic changes of IS with high frequency. Here, we propose and experimentally validate a label-free, volumetric, and automated assessment method for IS dynamics using a combinational approach of optical diffraction tomography and deep learning-based segmentation. The proposed method enables an automatic and quantitative spatiotemporal analysis of IS kinetics of morphological and biochemical parameters associated with IS dynamics, providing a new option for immunological research.

In this study, we report a label-free bone marrow white blood cell classification framework that captures the three-dimensional (3D) refractive index (RI) distributions of individual cells and analyzes with deep learning. Without using labeling or staining processes, 3D RI distributions of individual white blood cells were exploited for accurate profiling of their subtypes. Powered by deep learning, our method used the high-dimensional information of the WBC RI tomogram voxels and achieved high accuracy. The results show >99 % accuracy for the binary classification of myeloids and lymphoids and >96 % accuracy for the four-type classification of B, T lymphocytes, monocytes, and myelocytes. Furthermore, the feature learning of our approach was visualized via an unsupervised dimension reduction technique. We envision that this framework can be integrated into existing workflows for blood cell investigation, thereby providing cost-effective and rapid diagnosis of hematologic malignancy.

Simultaneous imaging of various facets of intact biological systems across multiple spatiotemporal scales would be an invaluable tool in biomedicine. However, conventional imaging modalities have stark tradeoffs precluding the fulfilment of all functional requirements. Here we propose the refractive index (RI), an intrinsic quantity governing light-matter interaction, as a means for such measurement. We show that major endogenous subcellular structures, which are conventionally accessed via exogenous fluorescence labeling, are encoded in 3D RI tomograms. We decode this information in a data-driven manner, thereby achieving multiplexed microtomography. This approach inherits the advantages of both high-specificity fluorescence imaging and label-free RI imaging. The performance, reliability, and scalability of this technology have been extensively characterized, and its application within single-cell profiling at unprecedented scales has been demonstrated.

In tomographic reconstruction, the image quality of the reconstructed images can be significantly degraded by defects in the measured two-dimensional (2D) raw image data. Despite the importance of screening defective 2D images for robust tomographic reconstruction, manual inspection and rule-based automation suffer from low-throughput and insufficient accuracy, respectively. Here, we present deep learning-enabled quality control for holographic data to produce robust and high-throughput optical diffraction tomography (ODT). The key idea is to distil the knowledge of an expert into a deep convolutional neural network. We built an extensive database of optical field images with clean/noisy annotations, and then trained a binary-classification network based upon the data. The trained network outperformed visual inspection by non-expert users and a widely used rule-based algorithm, with >90% test accuracy. Subsequently, we confirmed that the superior screening performance significantly improved the tomogram quality. To further confirm the trained model’s performance and generalisability, we evaluated it on unseen biological cell data obtained with a setup that was not used to generate the training dataset. Lastly, we interpreted the trained model using various visualisation techniques that provided the saliency map underlying each model inference. We envision the proposed network would a powerful lightweight module in the tomographic reconstruction pipeline.

The healthcare industry is in dire need for rapid microbial identification techniques. Microbial infection is a major healthcare issue with significant prevalence and mortality, which can be treated effectively during the early stages using appropriate antibiotics. However, determining the appropriate antibiotics for the treatment of the early stages of infection remains a challenge, mainly due to the lack of rapid microbial identification techniques. Conventional culture-based identification and matrix-assisted laser desorption/ionization time-of-flight mass spectroscopy are the gold standard methods, but the sample amplification process is extremely time-consuming. Here, we propose an identification framework that can be used to measure minute quantities of microbes by incorporating artificial neural networks with three-dimensional quantitative phase imaging. We aimed to accurately identify the species of bacterial bloodstream infection pathogens based on a single colony-forming unit of the bacteria. The successful distinction between a total of 19 species, with the accuracy of 99.9% when ten bacteria were measured, suggests that our framework can serve as an effective advisory tool for clinicians during the initial antibiotic prescription.Abstract Figure