Search

Clinical risk prediction requires comprehensive integration of clinical information and concurrent radiological findings. We present an upgraded chest radiograph (CXR) explainable artificial intelligence (xAI) model, which was trained on 241,723 well-annotated CXRs obtained prior to the onset of the COVID-19 pandemic. Mean area under the receiver operating characteristic curve (AUROC) for detection of 20 radiographic features was 0.955 (95% CI 0.938-0.955) on PA view and 0.909 (95% CI 0.890-0.925) on AP view. Coexistent and correlated radiographic findings are displayed in an interpretation table, and calibrated classifier confidence is displayed on an AI scoreboard. Retrieval of similar feature patches and comparable CXRs from a Model-Derived Atlas provides justification for model predictions. To demonstrate the feasibility of a fine-tuning approach for efficient and scalable development of xAI risk prediction models, we applied our CXR xAI model, in combination with clinical information, to predict oxygen requirement in COVID-19 patients. Prediction accuracy for high flow oxygen (HFO) and mechanical ventilation (MV) was 0.953 and 0.934 at 24 hours and 0.932 and 0.836 at 72 hours from the time of emergency department (ED) admission, respectively. Our CXR xAI model is auditable and captures key pathophysiological manifestations of cardiorespiratory diseases and cardiothoracic comorbidities. This model can be efficiently and broadly applied via a fine-tuning approach to provide fully automated risk and outcome predictions in various clinical scenarios in real-world practice.

Background Multicenter studies are required to validate the added benefit of using deep convolutional neural network (DCNN) software for detecting malignant pulmonary nodules on chest radiographs. Purpose To compare the performance of radiologists in detecting malignant pulmonary nodules on chest radiographs when assisted by deep learning-based DCNN software with that of radiologists or DCNN software alone in a multicenter setting. Materials and Methods Investigators at four medical centers retrospectively identified 600 lung cancer-containing chest radiographs and 200 normal chest radiographs. Each radiograph with a lung cancer had at least one malignant nodule confirmed by CT and pathologic examination. Twelve radiologists from the four centers independently analyzed the chest radiographs and marked regions of interest. Commercially available deep learning-based computer-aided detection software separately trained, tested, and validated with 19 330 radiographs was used to find suspicious nodules. The radiologists then reviewed the images with the assistance of DCNN software. The sensitivity and number of false-positive findings per image of DCNN software, radiologists alone, and radiologists with the use of DCNN software were analyzed by using logistic regression and Poisson regression. Results The average sensitivity of radiologists improved (from 65.1% [1375 of 2112; 95% confidence interval {CI}: 62.0%, 68.1%] to 70.3% [1484 of 2112; 95% CI: 67.2%, 73.1%], P < .001) and the number of false-positive findings per radiograph declined (from 0.2 [488 of 2400; 95% CI: 0.18, 0.22] to 0.18 [422 of 2400; 95% CI: 0.16, 0.2], P < .001) when the radiologists re-reviewed radiographs with the DCNN software. For the 12 radiologists in this study, 104 of 2400 radiographs were positively changed (from false-negative to true-positive or from false-positive to true-negative) using the DCNN, while 56 of 2400 radiographs were changed negatively. Conclusion Radiologists had better performance with deep convolutional network software for the detection of malignant pulmonary nodules on chest radiographs than without. © RSNA, 2019 Online supplemental material is available for this article. See also the editorial by Jacobson in this issue.

Despite the well-established impact of sex and sex hormones on bone structure and density, there has been limited description of sexual dimorphism in the hand and wrist in the literature. We developed a deep convolutional neural network (CNN) model to predict sex based on hand radiographs of children and adults aged between 5 and 70 years. Of the 1531 radiographs tested, the algorithm predicted sex correctly in 95.9% (κ = 0.92) of the cases. Two human radiologists achieved 58% (κ = 0.15) and 46% (κ = − 0.07) accuracy. The class activation maps (CAM) showed that the model mostly focused on the 2nd and 3rd metacarpal base or thumb sesamoid in women, and distal radioulnar joint, distal radial physis and epiphysis, or 3rd metacarpophalangeal joint in men. The radiologists reviewed 70 cases (35 females and 35 males) labeled with sex along with heat maps generated by CAM, but they could not find any patterns that distinguish the two sexes. A small sample of patients (n = 44) with sexual developmental disorders or transgender identity was selected for a preliminary exploration of application of the model. The model prediction agreed with phenotypic sex in only 77.8% (κ = 0.54) of these cases. To the best of our knowledge, this is the first study that demonstrated a machine learning model to perform a task in which human experts could not fulfill.

Recent advances and future perspectives of machine learning techniques offer promising applications in medical imaging. Machine learning has the potential to improve different steps of the radiology workflow including order scheduling and triage, clinical decision support systems, detection and interpretation of findings, postprocessing and dose estimation, examination quality control, and radiology reporting. In this article, the authors review examples of current applications of machine learning and artificial intelligence techniques in diagnostic radiology. In addition, the future impact and natural extension of these techniques in radiology practice are discussed. © RSNA, 2018

Pretreatment risk stratification is key for personalized medicine. While many physicians rely on an "eyeball test" to assess whether patients will tolerate major surgery or chemotherapy, "eyeballing" is inherently subjective and difficult to quantify. The concept of morphometric age derived from cross-sectional imaging has been found to correlate well with outcomes such as length of stay, morbidity, and mortality. However, the determination of the morphometric age is time intensive and requires highly trained experts. In this study, we propose a fully automated deep learning system for the segmentation of skeletal muscle cross-sectional area (CSA) on an axial computed tomography image taken at the third lumbar vertebra. We utilized a fully automated deep segmentation model derived from an extended implementation of a fully convolutional network with weight initialization of an ImageNet pre-trained model, followed by post processing to eliminate intramuscular fat for a more accurate analysis. This experiment was conducted by varying window level (WL), window width (WW), and bit resolutions in order to better understand the effects of the parameters on the model performance. Our best model, fine-tuned on 250 training images and ground truth labels, achieves 0.93 ± 0.02 Dice similarity coefficient (DSC) and 3.68 ± 2.29% difference between predicted and ground truth muscle CSA on 150 held-out test cases. Ultimately, the fully automated segmentation system can be embedded into the clinical environment to accelerate the quantification of muscle and expanded to volume analysis of 3D datasets.

Skeletal maturity progresses through discrete phases, a fact that is used routinely in pediatrics where bone age assessments (BAAs) are compared to chronological age in the evaluation of endocrine and metabolic disorders. While central to many disease evaluations, little has changed to improve the tedious process since its introduction in 1950. In this study, we propose a fully automated deep learning pipeline to segment a region of interest, standardize and preprocess input radiographs, and perform BAA. Our models use an ImageNet pretrained, fine-tuned convolutional neural network (CNN) to achieve 57.32 and 61.40% accuracies for the female and male cohorts on our held-out test images. Female test radiographs were assigned a BAA within 1 year 90.39% and within 2 years 98.11% of the time. Male test radiographs were assigned 94.18% within 1 year and 99.00% within 2 years. Using the input occlusion method, attention maps were created which reveal what features the trained model uses to perform BAA. These correspond to what human experts look at when manually performing BAA. Finally, the fully automated BAA system was deployed in the clinical environment as a decision supporting system for more accurate and efficient BAAs at much faster interpretation time (

Recent advancements in deep learning for automated image processing and classification have accelerated many new applications for medical image analysis. However, most deep learning algorithms have been developed using reconstructed, human-interpretable medical images. While image reconstruction from raw sensor data is required for the creation of medical images, the reconstruction process only uses a partial representation of all the data acquired. Here, we report the development of a system to directly process raw computed tomography (CT) data in sinogram-space, bypassing the intermediary step of image reconstruction. Two classification tasks were evaluated for their feasibility of sinogram-space machine learning: body region identification and intracranial hemorrhage (ICH) detection. Our proposed SinoNet, a convolutional neural network optimized for interpreting sinograms, performed favorably compared to conventional reconstructed image-space-based systems for both tasks, regardless of scanning geometries in terms of projections or detectors. Further, SinoNet performed significantly better when using sparsely sampled sinograms than conventional networks operating in image-space. As a result, sinogram-space algorithms could be used in field settings for triage (presence of ICH), especially where low radiation dose is desired. These findings also demonstrate another strength of deep learning where it can analyze and interpret sinograms that are virtually impossible for human experts.

PURPOSE: To investigate the diagnostic accuracy of cascading convolutional neural network (CNN) for urinary stone detection on unenhanced CT images and to evaluate the performance of pretrained models enriched with labeled CT images across different scanners. MATERIALS AND METHODS: This HIPAA-compliant, institutional review board–approved, retrospective clinical study used unenhanced abdominopelvic CT scans from 535 adults suspected of having urolithiasis. The scans were obtained on two scanners (scanner 1 [hereafter S1] and scanner 2 [hereafter S2]). A radiologist reviewed clinical reports and labeled cases for determination of reference standard. Stones were present on 279 (S1, 131; S2, 148) and absent on 256 (S1, 158; S2, 98) scans. One hundred scans (50 from each scanner) were randomly reserved as the test dataset, and the rest were used for developing a cascade of two CNNs: The first CNN identified the extent of the urinary tract, and the second CNN detected presence of stone. Nine variations of models were developed through the combination of different training data sources (S1, S2, or both [hereafter SB]) with (ImageNet, GrayNet) and without (Random) pretrained CNNs. First, models were compared for generalizability at the section level. Second, models were assessed by using area under the receiver operating characteristic curve (AUC) and accuracy at the patient level with test dataset from both scanners (n = 100). RESULTS: The GrayNet-pretrained model showed higher classifier exactness than did ImageNet-pretrained or Random-initialized models when tested by using data from the same or different scanners at section level. At the patient level, the AUC for stone detection was 0.92–0.95, depending on the model. Accuracy of GrayNet-SB (95%) was higher than that of ImageNet-SB (91%) and Random-SB (88%). For stones larger than 4 mm, all models showed similar performance (false-negative results: two of 34). For stones smaller than 4 mm, the number of false-negative results for GrayNet-SB, ImageNet-SB, and Random-SB were one of 16, three of 16, and five of 16, respectively. GrayNet-SB identified stones in all 22 test cases that had obstructive uropathy. CONCLUSION: A cascading model of CNNs can detect urinary tract stones on unenhanced CT scans with a high accuracy (AUC, 0.954). Performance and generalization of CNNs across scanners can be enhanced by using transfer learning with datasets enriched with labeled medical images. © RSNA, 2019 Supplemental material is available for this article. An earlier incorrect version appeared online. This article was corrected on August 6, 2019.

Artificial intelligence has been applied to many industries, including medicine. Among the various techniques in artificial intelligence, deep learning has attained the highest popularity in medical imaging in recent years. Many articles on deep learning have been published in radiologic journals. However, radiologists may have difficulty in understanding and interpreting these studies because the study methods of deep learning differ from those of traditional radiology. This review article aims to explain the concepts and terms that are frequently used in deep learning radiology articles, facilitating general radiologists' understanding.

Recently, considerable progress has been made in interpreting perceptual information through artificial intelligence, allowing better interpretation of highly complex data by machines. Furthermore, the applications of artificial intelligence, represented by deep learning technology, to the fields of medical and biomedical research are increasing exponentially. In this article, we will explain the stages of deep learning algorithm development in the field of medical imaging, namely topic selection, data collection, data exploration and refinement, algorithm development, algorithm evaluation, and clinical application; we will also discuss the latest trends for each stage.

Rationale and Objectives Cardiac computed tomography (CT) has emerged as a robust modality for imaging coronary stenosis and has recently been used to evaluate myocardial abnormalities such as ischemic perfusion defects and infarction. We developed a new image analysis algorithm for the semiautomatic and quantitative assessment of myocardial perfusion by CT. Materials and Methods The algorithm semiautomatically segments two-dimensional short-axis reformatted DICOM images of the left ventricle into regions of interest (ROIs) in accordance with American Heart Association (AHA) standards and is capable of creating nine further ROI subsegments. This includes separate endocardial, mid-ventricle, and epicardial layers. Image intensity values (Hounsfield unit) and relative myocardial thickness are quantitatively reported for each ROI and segment. Results The algorithm allows comparison of the HU values at the same ROI locations between rest and stress. The reproducibility is very good; ICC 0.89 for rest images, 0.83 for stress images. The mean time for generating ROIs for the entire heart was 11 minutes versus 22 minutes for manual tracing. Conclusion The algorithm reports parameters relevant for evaluation of stress perfusion CT studies and will allow more accurate and reproducible analysis in cardiac CT research.

To compare visualization of subtle normal and abnormal findings at computed tomography (CT) of the chest for diffuse lung disease with images reconstructed with filtered back projection and adaptive statistical iterative reconstruction (ASIR) techniques.In this HIPAA-compliant, institutional review board-approved study, 24 patients underwent 64-section multi-detector row CT of the chest for evaluation of diffuse lung disease. Scanning parameters included a pitch of 0.984:1 and 120 kVp in thin-section mode, with 2496 views per rotation compared with 984 views acquired for normal mode. The 0.625-mm-thick images were reconstructed with filtered back projection, ASIR, and ASIR high-definition (ASIR-HD) kernels. Two thoracic radiologists independently assessed the filtered back projection, ASIR, and ASIR-HD images for small anatomic details (interlobular septa, centrilobular region, and small bronchi and bronchioles), abnormal findings (reticulation, tiny nodules, altered attenuation, bronchiectasis), image quality (graded by using a six-point scale, where 1 = excellent image quality, and 5 = interpretation impossible), image noise, and artifacts. Data were tabulated for statistical testing.For visualization of normal and pathologic structures, CT image series reconstructed with ASIR-HD were rated substantially better than those reconstructed with filtered back projection and ASIR (P.001). ASIR-HD images were superior to filtered back projection images in 15 of 24 (62%) patients for visualization of normal structures and in 24 of 24 (100%) patients for pathologic findings. ASIR-HD was superior to ASIR in three of 24 (12%) images for normal anatomic findings and in seven of 24 (29%) images for pathologic evaluation. None of the images in the three groups were rated as unacceptable for noise (P.001).ASIR-HD reconstruction results in superior visualization of subtle and tiny anatomic structures and lesions in diffuse lung disease compared with ASIR and filtered back projection reconstructions.

Reliable computer-aided detection (CADe) of small polyps and flat lesions is limited by the relatively low image resolution of computed tomographic colonography (CTC). We developed a sinogram-based super-resolution (SR) method to enhance the images of lesion candidates detected by CADe. First, CADe is used to detect lesion candidates at high sensitivity from conventional CTC images. Next, the signal patterns of the lesion candidates are enhanced in sinogram domain by use of non-uniform compressive sampling and iterative reconstruction to produce SR images of the lesion candidates. For pilot evaluation, an anthropomorphic phantom including simulated lesions was filled partially with fecal tagging and scanned by use of a CT scanner. A fully automated CADe scheme was used to detect lesion candidates in the images reconstructed at conventional 0.61-mm and at 0.10-mm SR image resolution. The proof-of-concept results indicate that the SR method has potential to reduce the number of FP CADe detections below that obtainable with the conventional CTC imaging technology.

Today, while many researchers focus on the improvement of the regularization term in IR algorithms, they pay less concern to the improvement of the fidelity term. In this paper, we hypothesize that improving the fidelity term will further improve IR image quality in low-dose scanning, which typically causes more noise. The purpose of this paper is to systematically test and examine the role of high-fidelity system models using raw data in the performance of iterative image reconstruction approach minimizing energy functional. We first isolated the fidelity term and analyzed the importance of using focal spot area modeling, flying focal spot location modeling, and active detector area modeling as opposed to just flying focal spot motion. We then compared images using different permutations of all three factors. Next, we tested the ability of the fidelity terms to retain signals upon application of the regularization term with all three factors. We then compared the differences between images generated by the proposed method and Filtered-Back-Projection. Lastly, we compared images of low-dose in vivo data using Filtered-Back-Projection, Iterative Reconstruction in Image Space, and the proposed method using raw data. The initial comparison of difference maps of images constructed showed that the focal spot area model and the active detector area model also have significant impacts on the quality of images produced. Upon application of the regularization term, images generated using all three factors were able to substantially decrease model mismatch error, artifacts, and noise. When the images generated by the proposed method were tested, conspicuity greatly increased, noise standard deviation decreased by 90% in homogeneous regions, and resolution also greatly improved. In conclusion, the improvement of the fidelity term to model clinical scanners is essential to generating higher quality images in low-dose imaging.

Aims To compare the accuracy of two plaque delineation methods for coronary computed tomographic angiography (CTA) to identify lipid-core plaque (LCP) using histology as the reference standard. Methods and results Five ex vivo hearts were analysed by CTA and histology. LCP was defined by histology as fibroatheroma with core diameter/circumference >200 μm/>60° and cap thickness

An automated, computer-aided diagnosis (CAD) algorithm for the quantification of pneumothoraces from Multidetector Computed Tomography (MDCT) images has been developed. Algorithm performance was evaluated through comparison to manual segmentation by expert radiologists. A combination of two-dimensional and three-dimensional processing techniques was incorporated to reduce required processing time by two-thirds (as compared to similar techniques). Volumetric measurements on relative pneumothorax size were obtained and the overall performance of the automated method shows an average error of just below 1%.

Purpose We aimed to assess the temporal change in radiation doses from coronary computed tomography angiography (CCTA) during a 6-year period. High CCTA radiation doses have been reduced by multiple technologies that, if used appropriately, can decrease exposures significantly. Methods A total of 1277 examinations performed from 2005 to 2010 were included. Univariate and multivariable regression analysis of patient- and scan-related variables was performed with estimated radiation dose as the main outcome measure. Results Median doses decreased by 74.8% ( P 001), from 13.1 millisieverts (mSv) (interquartile range 9.3-14.7) in period 1 to 3.3 mSv (1.8-6.7) in period 4. Factors associated with greatest dose reductions ( P 001) were all most frequently applied in period 4: axial-sequential acquisition (univariate: −8.0 mSv [−9.7 to −7.9]), high-pitch helical acquisition (univariate: −8.8 mSv [−9.3 to −7.9]), reduced tube voltage (100 vs 120 kV) (univariate: −6.4 mSv [−7.4 to −5.4]), and use of automatic exposure control (univariate: −5.3 mSv [−6.2 to −4.4]). Conclusions CCTA radiation doses were reduced 74.8% through increasing use of dose-saving measures and evolving scanner technology.

Perfusion imaging is a useful adjunct to anatomic imaging in numerous diagnostic and therapy-monitoring settings. One approach to perfusion imaging is to assume a convolution relationship between a local arterial input function and the tissue enhancement profile of the region of interest via a ?residue function? and subsequently solve for this residue function. This ill-posed problem is generally solved using singular-value decomposition based approaches, and the hemodynamic parameters are solved for each voxel independently. In this paper, we present a formulation which incorporates both spatial and temporal correlations, and show through simulations that this new formulation yields higher accuracy and greater robustness with respect to image noise. We also show using rectal cancer tumor images that this new formulation results in better segregation of normal and cancerous voxels.

Although iterative reconstruction techniques (IRTs) have been shown to produce images of superior quality over conventional filtered back projection (FBP) based algorithms, the use of IRT in a clinical setting has been hampered by the significant computational demands of these algorithms. In this paper we present results of our efforts to overcome this hurdle by exploiting the combined computational power of multiple graphical processing units (GPUs). We have implemented forward and backward projection steps of reconstruction on an NVIDIA Tesla S870 hardware using CUDA. We have been able to accelerate forward projection by 71x and backward projection by 137x. We generate these results with no perceptible difference in image quality between the GPU and serial CPU implementations. This work illustrates the power of using commercial off-the-shelf relatively low-cost GPUs, potentially allowing IRT tomographic image reconstruction to be run in near real time, lowering the barrier to entry of IRT, and enabling deployment in the clinic.

Optimizing program execution targeted for Graphics Processing Units (GPUs) can be very challenging. Our ability to efficiently map serial code to a GPU or stream processing platform is a time consuming task and is greatly hampered by a lack of detail about the underlying hardware. Programmers are left to attempt trial and error to produce optimized codes.Recent publication of the underlying instruction set architecture (ISA) of the AMD/ATI GPU has allowed researchers to begin to propose aggressive optimizations. In this work, we present an optimization methodology that utilizes this information to accelerate programs on AMD/ATI GPUs. We start by defining optimization spaces that guide our work. We begin with disassembled machine code and collect program statistics provided by the AMD Graphics Shader Analyzer (GSA) profiling toolset. We explore optimizations targeting three different computing resources: 1) ALUs, 2) fetch bandwidth, and 3) thread usage, and present optimization techniques that consider how to better utilize each resource.We demonstrate the effectiveness of our proposed optimization approach on an AMD Radeon HD3870 GPU using the Brook+ stream programming language. We describe our optimizations using two commonly-used GPGPU applications that present very different program characteristics and optimization spaces: matrix multiplication and back-projection for medical image reconstruction. Our results show that optimized code can improve performance by 1.45x--6.7x as compared to unoptimized code run on the same GPU platform. The speedup obtained with our optimized implementations are 882x (matrix multiply) and 19x (back-projection) faster as compared with serial implementations run on an Intel 2.66 GHz Core 2 Duo with a 2 GB main memory.

This paper presents a novel segmentation methodology for automated classification and differentiation of soft tissues using multiband data obtained with the newly developed system of high-resolution ultrasonic transmission tomography (HUTT) for imaging biological organs. This methodology extends and combines two existing approaches: the L-level set active contour (AC) segmentation approach and the agglomerative hierarchical kappa-means approach for unsupervised clustering (UC). To prevent the trapping of the current iterative minimization AC algorithm in a local minimum, we introduce a multiresolution approach that applies the level set functions at successively increasing resolutions of the image data. The resulting AC clusters are subsequently rearranged by the UC algorithm that seeks the optimal set of clusters yielding the minimum within-cluster distances in the feature space. The presented results from Monte Carlo simulations and experimental animal-tissue data demonstrate that the proposed methodology outperforms other existing methods without depending on heuristic parameters and provides a reliable means for soft tissue differentiation in HUTT images.

In this paper, we are interested in soft tissue differentiation by multiband images obtained from the High-Resolution Ultrasonic Transmission Tomography (HUTT) system using a spectral target detection method based on constrained energy minimization (CEM). We have developed a new tissue differentiation method (called "CEM filter bank") consisting of multiple CEM filters specially designed for detecting multiple types of tissues. Statistical inference on the output of the CEM filter bank is used to make a decision based on the maximum statistical significance rather than the magnitude of each CEM filter output. We test and validate this method through three-dimensional interphantom/intraphantom soft tissue classification where target profiles obtained from an arbitrary single slice are used for differentiation over multiple other tomographic slices. The performance of the proposed classifier is assessed using receiver operating characteristic analysis. We also apply our method to classify tiny structures inside a bovine kidney and sheep kidneys. Using the proposed method we can detect physical objects and biological tissues such as styrofoam balls, chicken tissue, calyces, and vessel-duct successfully.

Current research on medical image processing relies heavily on the amount and quality of input data. Specifically, supervised machine learning methods require well-annotated datasets. A lack of annotation tools limits the potential to achieve high-volume processing and scaled systems with a proper reward mechanism. We developed MarkIt, a web-based tool, for collaborative annotation of medical imaging data with artificial intelligence and blockchain technologies. Our platform handles both Digital Imaging and Communications in Medicine (DICOM) and non-DICOM images, and allows users to annotate them for classification and object detection tasks in an efficient manner. MarkIt can accelerate the annotation process and keep track of user activities to calculate a fair reward. A proof-of-concept experiment was conducted with three fellowship-trained radiologists, each of whom annotated 1,000 chest X-ray studies for multi-label classification. We calculated the inter-rater agreement and estimated the value of the dataset to distribute the reward for annotators using a crypto currency. We hypothesize that MarkIt allows the typically arduous annotation task to become more efficient. In addition, MarkIt can serve as a platform to evaluate the value of data and trade the annotation results in a more scalable manner in the future. The platform is publicly available for testing on https://markit.mgh.harvard.edu.