Search

Purpose Aggressive posterior retinopathy of prematurity (AP-ROP) is a vision-threatening disease with a significant rate of progression to retinal detachment. The purpose of this study was to characterize AP-ROP quantitatively by demographics, rate of disease progression, and a deep learning-based vascular severity score. Design Retrospective analysis. Participants The Imaging and Informatics in ROP cohort from 8 North American centers, consisting of 947 patients and 5945 clinical eye examinations with fundus images, was used. Pretreatment eyes were categorized by disease severity: none, mild, type 2 or pre-plus, treatment-requiring (TR) without AP-ROP, TR with AP-ROP. Analyses compared TR with AP-ROP and TR without AP-ROP to investigate differences between AP-ROP and other TR disease. Methods A reference standard diagnosis was generated for each eye examination using previously published methods combining 3 independent image-based gradings and 1 ophthalmoscopic grading. All fundus images were analyzed using a previously published deep learning system and were assigned a score from 1 through 9. Main Outcome Measures Birth weight, gestational age, postmenstrual age, and vascular severity score. Results Infants who demonstrated AP-ROP were more premature by birth weight (617 g vs. 679 g; P = 0.01) and gestational age (24.3 weeks vs. 25.0 weeks; P Conclusions Premature infants in North America with AP-ROP are born younger and demonstrate disease earlier than infants with less severe ROP. Disease severity is quantifiable with a deep learning-based score, which correlates with clinically identified categories of disease, including AP-ROP. The rate of progression to peak disease is greatest in eyes that demonstrate AP-ROP compared with other treatment-requiring eyes. Analysis of quantitative characteristics of AP-ROP may help improve diagnosis and treatment of an aggressive, vision-threatening form of ROP.

Perceptual fading of an artificial scotoma can be viewed as a failure of figure–ground segregation, providing a useful tool for investigating possible mechanisms and processes involved in figure–ground perception. Weisstein’s antagonistic magnocellular/parvocellular stream figure–ground model proposes P stream activity encodes figure, and M stream activity encodes background. Where a boundary separates two regions, the region that is perceived as figure or ground is determined by the outcome of antagonism between M and P activity within each region and across the boundary between them. The region with the relatively stronger P “figure signal” is perceived as figure, and the region with the relatively stronger M “ground signal” is perceived as ground. From this perspective, fading occurs when the figure signal is overwhelmed by the ground signal. Strengthening the figure signal or weakening the ground signal should make the figure more resistant to fading. Based on research showing that red light suppresses M activity and short wavelength sensitive S-cones provide minimal input to M cells, we used red and blue light to reduce M activity in both figure and ground. The time to fade from stimulus onset until the figure completely disappeared was measured. Every combination of gray, green, red, and blue as figure and/or ground was tested. Compared with gray and green light, fade times were greatest when red or blue light either strengthened the figure signal by reducing M activity in the figure, or weakened the ground signal by reducing M activity in ground. The results support a dynamic antagonistic relationship between M and P activity contributing to figure–ground perception as envisioned in Weisstein’s model.

Translating deep learning research from theory into clinical practice has unique challenges, specifically in the field of neuroimaging. In this paper, we present DeepNeuro, a Python-based deep learning framework that puts deep neural networks for neuroimaging into practical usage with a minimum of friction during implementation. We show how this framework can be used to design deep learning pipelines that can load and preprocess data, design and train various neural network architectures, and evaluate and visualize the results of trained networks on evaluation data. We present a way of reproducibly packaging data pre- and postprocessing functions common in the neuroimaging community, which facilitates consistent performance of networks across variable users, institutions, and scanners. We show how deep learning pipelines created with DeepNeuro can be concisely packaged into shareable Docker and Singularity containers with user-friendly command-line interfaces.

Numerous studies have shown that more visual features can be stored in visual short-term memory (VSTM) when they are encoded from fewer objects (Luck & Vogel, 1997, Nature, 390, 279-281; Olson & Jiang, 2002, Perception & Psychophysics, 64[7], 1055-1067). This finding has been consistent for simple objects with one surface and one boundary contour, but very few experiments have shown a clear performance benefit when features are organized as multipart objects versus spatially dispersed single-feature objects. Some researchers have suggested multipart object integration is not mandatory because of the potential ambiguity of the display (Balaban & Luria, 2015, Cortex, 26(5), 2093-2104; Luria & Vogel, 2014, Journal of Cognitive Neuroscience, 26[8], 1819-1828). For example, a white bar across the middle of a red circle could be interpreted as two objects, a white bar occluding a red circle, or as a single two-colored object. We explore whether an object benefit can be found by disambiguating the figure-ground organization of multipart objects using a luminance gradient and linear perspective to create the appearance of a unified surface. Also, we investigated memory for objects with a visual feature indicated by a hole, rather than an additional surface on the object. Results indicate the organization of multipart objects can influence VSTM performance, but the effect is driven by how the specific organization allows for use of global ensemble statistics of the memory array rather than a memory benefit for local object representations.

Retrospective evaluation of a deep learning-derived retinopathy of prematurity (ROP) vascular severity score in an operational ROP screening program demonstrated high diagnostic performance for detection of type 2 or worse ROP. To our knowledge, this is the first report in the literature that evaluated the use of artificial intelligence for ROP screening and represents a proof of concept. With further prospective validation, this technology might improve the accuracy, efficiency, and objectivity of diagnosis and facilitate earlier detection of disease progression in patients with potentially blinding ROP.

Background Longitudinal measurement of glioma burden with MRI is the basis for treatment response assessment. In this study, we developed a deep learning algorithm that automatically segments abnormal fluid attenuated inversion recovery (FLAIR) hyperintensity and contrast-enhancing tumor, quantitating tumor volumes as well as the product of maximum bidimensional diameters according to the Response Assessment in Neuro-Oncology (RANO) criteria (AutoRANO). Methods Two cohorts of patients were used for this study. One consisted of 843 preoperative MRIs from 843 patients with low- or high-grade gliomas from 4 institutions and the second consisted of 713 longitudinal postoperative MRI visits from 54 patients with newly diagnosed glioblastomas (each with 2 pretreatment “baseline” MRIs) from 1 institution. Results The automatically generated FLAIR hyperintensity volume, contrast-enhancing tumor volume, and AutoRANO were highly repeatable for the double-baseline visits, with an intraclass correlation coefficient (ICC) of 0.986, 0.991, and 0.977, respectively, on the cohort of postoperative GBM patients. Furthermore, there was high agreement between manually and automatically measured tumor volumes, with ICC values of 0.915, 0.924, and 0.965 for preoperative FLAIR hyperintensity, postoperative FLAIR hyperintensity, and postoperative contrast-enhancing tumor volumes, respectively. Lastly, the ICCs for comparing manually and automatically derived longitudinal changes in tumor burden were 0.917, 0.966, and 0.850 for FLAIR hyperintensity volume, contrast-enhancing tumor volume, and RANO measures, respectively. Conclusions Our automated algorithm demonstrates potential utility for evaluating tumor burden in complex posttreatment settings, although further validation in multicenter clinical trials will be needed prior to widespread implementation.

Utilizing 23 datasets, we report a meta-analysis of an asymmetry in presaccadic fixation durations for saccades directed above and below eye fixation during saccadic exploration. For inclusion in the meta-analysis, saccadic exploration of complex visual displays had to have been made without gaze-contingent manipulations. Effect sizes for the asymmetry were quantified as Hedge’s g. Pooled effect sizes indicated significant asymmetries such that during saccadic exploration in a variety of tasks, presaccadic fixation durations for saccades directed into the upper visual field were reliably shorter than presaccadic fixation durations for saccades into the lower visual field. It is contended that the asymmetry is robust and important for efforts aimed at modelling when a saccade is initiated as a function of ensuing saccade direction.

Advanced 3D imaging modalities, such as micro-computed tomography (micro-CT), have been incorporated into the high-throughput embryo pipeline of the International Mouse Phenotyping Consortium (IMPC). This project generates large volumes of raw data that cannot be immediately exploited without significant resources of personnel and expertise. Thus, rapid automated annotation is crucial to ensure that 3D imaging data can be integrated with other multi-dimensional phenotyping data. We present an automated computational mouse embryo phenotyping pipeline that harnesses the large amount of wild-type control data available in the IMPC embryo pipeline in order to address issues of low mutant sample number as well as incomplete penetrance and variable expressivity. We also investigate the effect of developmental substage on automated phenotyping results. Designed primarily for developmental biologists, our software performs image pre-processing, registration, statistical analysis and segmentation of embryo images. We also present a novel anatomical E14.5 embryo atlas average and, using it with LAMA, show that we can uncover known and novel dysmorphology from two IMPC knockout lines., Summary: Introducing an easy-to-use automated anatomical phenotyping pipeline for mouse embryos along with a highly-detailed anatomical E14.5 atlas.

Background Prince George's County Maryland, historically a medically underserved region, has a population of 909 327 and a high incidence of cardiometabolic syndrome and hypertension. Application of level I evidence practices in such areas requires the availability of highly advanced cardiovascular interventions. Donabedian principles of quality of care were applied to a failing cardiac surgery program. We hypothesized that a multidisciplinary application of this model supported by partnership with a university hospital system could result in improved quality care outcomes. Methods and Results A 6‐month assessment and planning process commenced in July 2014. Preoperative, intraoperative, and postoperative protocols were developed before program restart. Staff education and training was conducted via team simulation and rehearsal sessions. A total of 425 patients underwent cardiac surgical procedures. Quality tracking of key performance measures was conducted, and 323 isolated coronary artery bypass grafting procedures were performed from July 2014 to December 2019. Key risk factors in our patient demographic were higher than the Society of Thoracic Surgeons national mean. Risk‐adjusted outcome data yielded a mortality rate of 0.3% versus 2.2% nationally. The overall major complication rate was lower than expected at 7.1% compared with 11.5% nationally. Readmission rate was less than the Society of Thoracic Surgeons mean for isolated coronary artery bypass grafting (4.0% versus 10.1%, P Conclusions Excellent outcomes in cardiac surgery are attainable following program renovation in an underserved region in the setting of low volume. The principles and processes applied have potential broad application for any quality improvement effort.

Purpose To evaluate the clinical usefulness of a quantitative deep learning-derived vascular severity score for retinopathy of prematurity (ROP) by assessing its correlation with clinical ROP diagnosis and by measuring clinician agreement in applying a novel scale. Design Analysis of existing database of posterior pole fundus images and corresponding ophthalmoscopic examinations using 2 methods of assigning a quantitative scale to vascular severity. Participants Images were from clinical examinations of patients in the Imaging and Informatics in ROP Consortium. Four ophthalmologists and 1 study coordinator evaluated vascular severity on a scale from 1 to 9. Methods A quantitative vascular severity score (1–9) was applied to each image using a deep learning algorithm. A database of 499 images was developed for assessment of interobserver agreement. Main Outcome Measures Distribution of deep learning-derived vascular severity scores with the clinical assessment of zone (I, II, or III), stage (0, 1, 2, or 3), and extent (6 clock hours) of stage 3 evaluated using multivariate linear regression and weighted κ values and Pearson correlation coefficients for interobserver agreement on a 1-to-9 vascular severity scale. Results For deep learning analysis, a total of 6344 clinical examinations were analyzed. A higher deep learning-derived vascular severity score was associated with more posterior disease, higher disease stage, and higher extent of stage 3 disease (P < 0.001 for all). For a given ROP stage, the vascular severity score was higher in zone I than zones II or III (P < 0.001). Multivariate regression found zone, stage, and extent all were associated independently with the severity score (P < 0.001 for all). For interobserver agreement, the mean ± standard deviation weighted κ value was 0.67 ± 0.06, and the Pearson correlation coefficient ± standard deviation was 0.88 ± 0.04 on the use of a 1-to-9 vascular severity scale. Conclusions A vascular severity scale for ROP seems feasible for clinical adoption; corresponds with zone, stage, extent of stage 3, and plus disease; and facilitates the use of objective technology such as deep learning to improve the consistency of ROP diagnosis.

Advanced 3D imaging modalities such as micro computed tomography (micro-CT), high resolution episcopic microscopy (HREM), and optical projection tomography (OPT) have been readily incorporated into high-throughput phenotyping pipelines, such as the International Mouse Phenotyping Consortium (IMPC). Such modalities generate large volumes of raw data that cannot be immediately harnessed without significant resources of manpower and expertise. Thus, rapid automated analysis and annotation is critical to ensure that 3D imaging data is able to be integrated with other multi-dimensional phenotyping data. To this end, we present an automated computational mouse phenotyping pipeline called LAMA, based on image registration, which requires minimal technical expertise and human input to use. Designed predominantly for developmental biologists, our software performs image pre-processing, registration, statistical and gene function annotation, and segmentation of 3D micro-CT data. We address several limitations of current methods and create an easy to use, fast solution application for mouse embryo phenotyping. We also present a highly granular, novel anatomical E14.5 (14.5 days post coitus) atlas of a population average that integrates with our pipeline to allow a range of dysmorphologies to be automatically annotated as well as results from the validation of the pipeline.

Using medical images to evaluate disease severity and change over time is a routine and important task in clinical decision making. Grading systems are often used, but are unreliable as domain experts disagree on disease severity category thresholds. These discrete categories also do not reflect the underlying continuous spectrum of disease severity. To address these issues, we developed a convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum. We demonstrate this in two medical imaging domains: retinopathy of prematurity (ROP) in retinal photographs and osteoarthritis in knee radiographs. Our patient cohorts consist of 4861 images from 870 patients in the Imaging and Informatics in Retinopathy of Prematurity (i-ROP) cohort study and 10,012 images from 3021 patients in the Multicenter Osteoarthritis Study (MOST), both of which feature longitudinal imaging data. Multiple expert clinician raters ranked 100 retinal images and 100 knee radiographs from excluded test sets for severity of ROP and osteoarthritis, respectively. The Siamese neural network output for each image in comparison to a pool of normal reference images correlates with disease severity rank (ρ = 0.87 for ROP and ρ = 0.89 for osteoarthritis), both within and between the clinical grading categories. Thus, this output can represent the continuous spectrum of disease severity at any single time point. The difference in these outputs can be used to show change over time. Alternatively, paired images from the same patient at two time points can be directly compared using the Siamese neural network, resulting in an additional continuous measure of change between images. Importantly, our approach does not require manual localization of the pathology of interest and requires only a binary label for training (same versus different). The location of disease and site of change detected by the algorithm can be visualized using an occlusion sensitivity map-based approach. For a longitudinal binary change detection task, our Siamese neural networks achieve test set receiving operator characteristic area under the curves (AUCs) of up to 0.90 in evaluating ROP or knee osteoarthritis change, depending on the change detection strategy. The overall performance on this binary task is similar compared to a conventional convolutional deep-neural network trained for multi-class classification. Our results demonstrate that convolutional Siamese neural networks can be a powerful tool for evaluating the continuous spectrum of disease severity and change in medical imaging.

Purpose Retinopathy of prematurity (ROP), a leading cause of childhood blindness, is diagnosed by clinical ophthalmoscopic examinations or reading retinal images. Plus disease, defined as abnormal tortuosity and dilation of the posterior retinal blood vessels, is the most important feature to determine treatment-requiring ROP. We aimed to create a complete, publicly available and feature-extraction-based pipeline, I-ROP ASSIST, that achieves convolutional neural network (CNN)-like performance when diagnosing plus disease from retinal images. Methods We developed two datasets containing 100 and 5512 posterior retinal images, respectively. After segmenting retinal vessels, we detected the vessel centerlines. Then, we extracted features relevant to ROP, including tortuosity and dilation measures, and used these features in the classifiers including logistic regression, support vector machine and neural networks to assess a severity score for the input. We tested our system with fivefold cross-validation and calculated the area under the curve (AUC) metric for each classifier and dataset. Results For predicting plus versus not-plus categories, we achieved 99% and 94% AUC on the first and second datasets, respectively. For predicting pre-plus or worse versus normal categories, we achieved 99% and 88% AUC on the first and second datasets, respectively. The CNN method achieved 98% and 94% for predicting two categories on the second dataset. Conclusions Our system combining automatic retinal vessel segmentation, tracing, feature extraction and classification is able to diagnose plus disease in ROP with CNN-like performance. Translational Relevance The high performance of I-ROP ASSIST suggests potential applications in automated and objective diagnosis of plus disease.

Purpose Retinopathy of prematurity is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective, which leads to treatment differences. Our goal was to determine objective differences in the diagnosis of plus disease between clinicians using an automated retinopathy of prematurity (ROP) vascular severity score. Design This retrospective cohort study used data from the Imaging and Informatics in ROP Consortium, which comprises 8 tertiary care centers in North America. Fundus photographs of all infants undergoing ROP screening examinations between July 1, 2011, and December 31, 2016, were obtained. Participants Infants meeting ROP screening criteria who were diagnosed with plus disease and treatment initiated by an examining physician based on ophthalmoscopic examination results. Methods An ROP severity score (1–9) was generated for each image using a deep learning (DL) algorithm. Main Outcome Measures The mean, median, and range of ROP vascular severity scores overall and for each examiner when the diagnosis of plus disease was made. Results A total of 5255 clinical examinations in 871 babies were analyzed. Of these, 168 eyes were diagnosed with plus disease by 11 different examiners and were included in the study. The mean ± standard deviation vascular severity score for patients diagnosed with plus disease was 7.4 ± 1.9, median was 8.5 (interquartile range, 5.8–8.9), and range was 1.1 to 9.0. Within some examiners, variability in the level of vascular severity diagnosed as plus disease was present, and 1 examiner routinely diagnosed plus disease in patients with less severe disease than the other examiners (P Conclusions We observed variability both between and within examiners in the diagnosis of plus disease using DL. Prospective evaluation of clinical trial data using an objective measurement of vascular severity may help to define better the minimum necessary level of vascular severity for the diagnosis of plus disease or how other clinical features such as zone, stage, and extent of peripheral disease ought to be incorporated in treatment decisions.

Objective Deep learning has become a promising approach for automated support for clinical diagnosis. When medical data samples are limited, collaboration among multiple institutions is necessary to achieve high algorithm performance. However, sharing patient data often has limitations due to technical, legal, or ethical concerns. In this study, we propose methods of distributing deep learning models as an attractive alternative to sharing patient data. Methods We simulate the distribution of deep learning models across 4 institutions using various training heuristics and compare the results with a deep learning model trained on centrally hosted patient data. The training heuristics investigated include ensembling single institution models, single weight transfer, and cyclical weight transfer. We evaluated these approaches for image classification in 3 independent image collections (retinal fundus photos, mammography, and ImageNet). Results We find that cyclical weight transfer resulted in a performance that was comparable to that of centrally hosted patient data. We also found that there is an improvement in the performance of cyclical weight transfer heuristic with a high frequency of weight transfer. Conclusions We show that distributing deep learning models is an effective alternative to sharing patient data. This finding has implications for any collaborative deep learning study.

Background Variation in red blood cell (RBC) transfusion practices exists at cardiac surgery centers across the nation. We tested the hypothesis that significant variation in RBC transfusion practices between centers in our state's cardiac surgery quality collaborative remains even after risk adjustment. Methods Using a multiinstitutional statewide database created by the Maryland Cardiac Surgery Quality Initiative (MCSQI), we included patient-level data from 8,141 patients undergoing isolated coronary artery bypass (CAB) or aortic valve replacement at 1 of 10 centers. Risk-adjusted multivariable logistic regression models were constructed to predict the need for any intraoperative RBC transfusion, as well as for any postoperative RBC transfusion, with anonymized center number included as a factor variable. Results Unadjusted intraoperative RBC transfusion probabilities at the 10 centers ranged from 13% to 60%; postoperative RBC transfusion probabilities ranged from 16% to 41%. After risk adjustment with demographic, comorbidity, and operative data, significant intercenter variability was documented (intraoperative probability range, 4% –59%; postoperative probability range, 13%–39%). When stratifying patients by preoperative hematocrit quartiles, significant variability in intraoperative transfusion probability was seen among all quartiles (lowest quartile: mean hematocrit value, 30.5% ± 4.1%, probability range, 17%–89%; highest quartile: mean hematocrit value, 44.8% ± 2.5%; probability range, 1%–35%). Conclusions Significant variation in intercenter RBC transfusion practices exists for both intraoperative and postoperative transfusions, even after risk adjustment, among our state's centers. Variability in intraoperative RBC transfusion persisted across quartiles of preoperative hematocrit values.

PURPOSE: To determine the influence of preprocessing on the repeatability and redundancy of radiomics features extracted using a popular open-source radiomics software package in a scan-rescan glioblastoma MRI study. MATERIALS AND METHODS: In this study, a secondary analysis of T2-weighted fluid-attenuated inversion recovery (FLAIR) and T1-weighted postcontrast images from 48 patients (mean age, 56 years [range, 22–77 years]) diagnosed with glioblastoma were included from two prospective studies (ClinicalTrials.gov NCT00662506 [2009–2011] and NCT00756106 [2008–2011]). All patients underwent two baseline scans 2–6 days apart using identical imaging protocols on 3-T MRI systems. No treatment occurred between scan and rescan, and tumors were essentially unchanged visually. Radiomic features were extracted by using PyRadiomics (https://pyradiomics.readthedocs.io/) under varying conditions, including normalization strategies and intensity quantization. Subsequently, intraclass correlation coefficients were determined between feature values of the scan and rescan. RESULTS: Shape features showed a higher repeatability than intensity (adjusted P < .001) and texture features (adjusted P < .001) for both T2-weighted FLAIR and T1-weighted postcontrast images. Normalization improved the overlap between the region of interest intensity histograms of scan and rescan (adjusted P < .001 for both T2-weighted FLAIR and T1-weighted postcontrast images), except in scans where brain extraction fails. As such, normalization significantly improves the repeatability of intensity features from T2-weighted FLAIR scans (adjusted P = .003 [z score normalization] and adjusted P = .002 [histogram matching]). The use of a relative intensity binning strategy as opposed to default absolute intensity binning reduces correlation between gray-level co-occurrence matrix features after normalization. CONCLUSION: Both normalization and intensity quantization have an effect on the level of repeatability and redundancy of features, emphasizing the importance of both accurate reporting of methodology in radiomics articles and understanding the limitations of choices made in pipeline design. Supplemental material is available for this article. © RSNA, 2020 See also the commentary by Tiwari and Verma in this issue.

IMPORTANCE: Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but clinical diagnosis is subjective and qualitative. OBJECTIVE: To describe a quantitative ROP severity score derived using a deep learning algorithm designed to evaluate plus disease and to assess its utility for objectively monitoring ROP progression. DESIGN, SETTING, AND PARTICIPANTS: This retrospective cohort study included images from 5255 clinical examinations of 871 premature infants who met the ROP screening criteria of the Imaging and Informatics in ROP (i-ROP) Consortium, which comprises 9 tertiary care centers in North America, from July 1, 2011, to December 31, 2016. Data analysis was performed from July 2017 to May 2018. EXPOSURE: A deep learning algorithm was used to assign a continuous ROP vascular severity score from 1 (most normal) to 9 (most severe) at each examination based on a single posterior photograph compared with a reference standard diagnosis (RSD) simplified into 4 categories: no ROP, mild ROP, type 2 ROP or pre-plus disease, or type 1 ROP. Disease course was assessed longitudinally across multiple examinations for all patients. MAIN OUTCOMES AND MEASURES: Mean ROP vascular severity score progression over time compared with the RSD. RESULTS: A total of 5255 clinical examinations from 871 infants (mean [SD] gestational age, 27.0 [2.0] weeks; 493 [56.6%] male; mean [SD] birth weight, 949 [271] g) were analyzed. The median severity scores for each category were as follows: 1.1 (interquartile range [IQR], 1.0-1.5) (no ROP), 1.5 (IQR, 1.1-3.4) (mild ROP), 4.6 (IQR, 2.4-5.3) (type 2 and pre-plus), and 7.5 (IQR, 5.0-8.7) (treatment-requiring ROP) (P

IMPORTANCE: Retinopathy of prematurity (ROP) is a leading cause of childhood blindness worldwide, but treatment failure and disease recurrence are important causes of adverse outcomes in patients with treatment-requiring ROP (TR-ROP). OBJECTIVES: To apply an automated ROP vascular severity score obtained using a deep learning algorithm and to assess its utility for objectively monitoring ROP regression after treatment. DESIGN, SETTING, AND PARTICIPANTS: This retrospective cohort study used data from the Imaging and Informatics in ROP consortium, which comprises 9 tertiary referral centers in North America that screen high volumes of at-risk infants for ROP. Images of 5255 clinical eye examinations from 871 infants performed between July 2011 and December 2016 were assessed for eligibility in the present study. The disease course was assessed with time across the numerous examinations for patients with TR-ROP. Infants born prematurely meeting screening criteria for ROP who developed TR-ROP and who had images captured within 4 weeks before and after treatment as well as at the time of treatment were included. MAIN OUTCOMES AND MEASURES: The primary outcome was mean (SD) ROP vascular severity score before, at time of, and after treatment. A deep learning classifier was used to assign a continuous ROP vascular severity score, which ranged from 1 (normal) to 9 (most severe), at each examination. A secondary outcome was the difference in ROP vascular severity score among eyes treated with laser or the vascular endothelial growth factor antagonist bevacizumab. Differences between groups for both outcomes were assessed using unpaired 2-tailed t tests with Bonferroni correction. RESULTS: Of 5255 examined eyes, 91 developed TR-ROP, of which 46 eyes met the inclusion criteria based on the available images. The mean (SD) birth weight of those patients was 653 (185) g, with a mean (SD) gestational age of 24.9 (1.3) weeks. The mean (SD) ROP vascular severity scores significantly increased 2 weeks prior to treatment (4.19 [1.75]), peaked at treatment (7.43 [1.89]), and decreased for at least 2 weeks after treatment (4.00 [1.88]) (all P

Introduction: Brain imaging is a key step in the clinical evaluation of ischemic stroke, with Diffusion Weighted Magnetic Resonance (DWI) being a key imaging modality, as it allows for assessment of extent of acute ischemic brain injury. Manual delineation of stroke regions is expensive, time-consuming, and subject to inter-rater variability. In this study, we develop a deep learning approach for ischemic stroke volumetric segmentation in a large clinical dataset of 1,239 patients from the NIH-funded Heart-Brain Interactions in Human Acute Ischemic Stroke Study utilizing only DWI imaging. Hypothesis: Deep learning can be used to automatically calculate stroke volumes in high agreement with manual human expert segmentations. Methods: The patients were randomly divided into Training (n = 743), Validation (n=248), and Testing (n=248). We implemented the 3D U-Net neural network architecture. Additionally, we modified the 3D U-Net by incorporating incorporate state-of-the-art components that have improved neural network architectures for classification tasks, namely residual connections, inception modules, dense connections, and squeeze-and-excitation modules. Results: The best performing individual model was the Inception U-Net, which had a median dice similarity coefficient of 0.720 (0.011-0.920) within the Testing Set. In comparing manually and automatically derived infarct volumes, the Intraclass Correlation Coefficient was 0.974 (p Conclusions: Our fully-automatic pipeline for stroke segmentation demonstrates the potential for deep learning-based tools to automate ischemic stroke volumetrics.

Introduction: Acute brain infarcts can provoke functional and morphologic changes in internal organs in the absence of primary visceral causes. We tested the hypothesis that infarcts in certain regions of the brain were much more likely to be associated with internal organ injury (IOI) than other regions using a method that was free from the bias of a-priori hypothesis as to any specific location. Methods: We generated neuroanatomic maps of brain infarcts for plasma troponin T elevation (structural cardiac marker), QTc prolongation on ECG (electrophysiological cardiac marker), pneumonia and urinary tract infection (marker for altered immune/pulmonary/urinary physiology), acute stress hyperglycemia (hepatic marker of glycogenolysis) in 1208 consecutive patients who were prospectively recruited into the NIH-funded HBI study. We utilized Threshold-Free Cluster Enhancement to examine the relationship between infarct location and IOI. We generated voxel-wise p-value maps using a permutation-based approach and identified clusters of contiguous voxels associated with IOI with a p-value Results: We identified significant clusters of voxels for each form of IOI. The clusters for troponin elevation and QTc prolongation encompassed similar regions in the right insular cortex. The cluster for acute stress hyperglycemia involved the left insula. We found two large clusters for post-stroke infection, one for pneumonia in the left and one for urinary tract infection in the right hemisphere, both encompassing the insular cortex. The relationship between infarct location and IOI persisted after adjusting for infarct volume, vascular risk factors, and stroke etiology. Conclusion: Our results uncover the neuroanatomical substrate of post-stroke IOI. Localizing discrete regions of brain infarcts associated with IOI could be used to bootstrap towards new markers for better differentiation between neurogenic and non-neurogenic mechanisms of post-stroke IOI.

Purpose Accurate image-based ophthalmic diagnosis relies on fundus image clarity. This has important implications for the quality of ophthalmic diagnoses and for emerging methods such as telemedicine and computer-based image analysis. The purpose of this study was to implement a deep convolutional neural network (CNN) for automated assessment of fundus image quality in retinopathy of prematurity (ROP). Design Experimental study. Participants Retinal fundus images were collected from preterm infants during routine ROP screenings. Methods Six thousand one hundred thirty-nine retinal fundus images were collected from 9 academic institutions. Each image was graded for quality (acceptable quality [AQ], possibly acceptable quality [PAQ], or not acceptable quality [NAQ]) by 3 independent experts. Quality was defined as the ability to assess an image confidently for the presence of ROP. Of the 6139 images, NAQ, PAQ, and AQ images represented 5.6%, 43.6%, and 50.8% of the image set, respectively. Because of low representation of NAQ images in the data set, images labeled NAQ were grouped into the PAQ category, and a binary CNN classifier was trained using 5-fold cross-validation on 4000 images. A test set of 2109 images was held out for final model evaluation. Additionally, 30 images were ranked from worst to best quality by 6 experts via pairwise comparisons, and the CNN’s ability to rank quality, regardless of quality classification, was assessed. Main Outcome Measures The CNN performance was evaluated using area under the receiver operating characteristic curve (AUC). A Spearman’s rank correlation was calculated to evaluate the overall ability of the CNN to rank images from worst to best quality as compared with experts. Results The mean AUC for 5-fold cross-validation was 0.958 (standard deviation, 0.005) for the diagnosis of AQ versus PAQ images. The AUC was 0.965 for the test set. The Spearman’s rank correlation coefficient on the set of 30 images was 0.90 as compared with the overall expert consensus ranking. Conclusions This model accurately assessed retinal fundus image quality in a comparable manner with that of experts. This fully automated model has potential for application in clinical settings, telemedicine, and computer-based image analysis in ROP and for generalizability to other ophthalmic diseases.

We consider learning from comparison labels generated as follows: given two samples in a dataset, a labeler produces a label indicating their relative order. Such comparison labels scale quadratically with the dataset size; most importantly, in practice, they often exhibit lower variance compared to class labels. We propose a new neural network architecture based on siamese networks to incorporate both class and comparison labels in the same training pipeline, using Bradley-Terry and Thurstone loss functions. Our architecture leads to a significant improvement in predicting both class and comparison labels, increasing classification AUC by as much as 35% and comparison AUC by as much as 6% on several real-life datasets. We further show that, by incorporating comparisons, training from few samples becomes possible: a deep neural network of 5.9 million parameters trained on 80 images attains a 0.92 AUC when incorporating comparisons.

Biofilms are a ubiquitous form of growth for bacteria on surfaces in most environments, natural or manmade. Here we present a protocol using the Bioflux microfluidic system to investigate the organized structure and development of these multicellular communities. Microfluidic systems present an opportunity to grow biofilms in a stable, physiologically maintained environment that is readily observable via time-lapse microscopy.

[Figure: see text] OBJECTIVES: Childhood blindness from retinopathy of prematurity (ROP) is increasing as a result of improvements in neonatal care worldwide. We evaluate the effectiveness of artificial intelligence (AI)–based screening in an Indian ROP telemedicine program and whether differences in ROP severity between neonatal care units (NCUs) identified by using AI are related to differences in oxygen-titrating capability. METHODS: External validation study of an existing AI-based quantitative severity scale for ROP on a data set of images from the Retinopathy of Prematurity Eradication Save Our Sight ROP telemedicine program in India. All images were assigned an ROP severity score (1–9) by using the Imaging and Informatics in Retinopathy of Prematurity Deep Learning system. We calculated the area under the receiver operating characteristic curve and sensitivity and specificity for treatment-requiring retinopathy of prematurity. Using multivariable linear regression, we evaluated the mean and median ROP severity in each NCU as a function of mean birth weight, gestational age, and the presence of oxygen blenders and pulse oxygenation monitors. RESULTS: The area under the receiver operating characteristic curve for detection of treatment-requiring retinopathy of prematurity was 0.98, with 100% sensitivity and 78% specificity. We found higher median (interquartile range) ROP severity in NCUs without oxygen blenders and pulse oxygenation monitors, most apparent in bigger infants (>1500 g and 31 weeks’ gestation: 2.7 [2.5–3.0] vs 3.1 [2.4–3.8]; P = .007, with adjustment for birth weight and gestational age). CONCLUSIONS: Integration of AI into ROP screening programs may lead to improved access to care for secondary prevention of ROP and may facilitate assessment of disease epidemiology and NCU resources.

Immune checkpoint inhibitors (ICI) have recently been shown to be effective for brain metastases (BM) in melanoma and lung cancer. However, accurately assessing intracranial response in patients undergoing ICI is a challenge, as current measures cannot reliably distinguish pseudoprogression from true tumor progression. To identify potential biomarkers of response, we analyzed standard post-contrast and dynamic susceptibility contrast MRI to identify characteristic vascular signatures as part of an ongoing Phase 2 study of pembrolizumab for patients with untreated or progressive, previously treated BM from any histology. Tumor volume measurements were calculated by summating all enhancing voxels. A volumetric increase of >40% was categorized as progressive disease (PD), a decrease of >60% as partial response (PR), and stable disease (SD) as between -60% and +40%. 78 patients have been enrolled, of whom 44 have received at least baseline advanced MR imaging. Histologies include 21 with breast cancer, 5 with non-small cell lung cancer, 4 with melanoma, and 13 with other cancers. At baseline, the total number of BM was 1-50+ per patient. Based on summing the entire enhancing intracranial disease burden, best volumetric responses for the 33 evaluable patients include 4 PR, 10 SD, and 19 PD. On preliminary analysis, there was a correlation between increased tumor cerebral blood volume/flow with tumor progression. Correlation of additional vascular physiologic parameters (e.g. vessel caliber, tissue oxygenation) to volumetric response, patient outcome, and standardized response criteria (iRANO) are ongoing. Our data provides potential evidence that effective ICI is associated with a decrease in perfusion. Ongoing analyses to uncover additional vascular changes – specifically longitudinal metrics reflecting vascular structure and function - within BM to ICI are pending. These findings have potential to explore mechanisms of ICI response and resistance, as well as biomarkers of response.

BackgroundPrior work has demonstrated the near-perfect accuracy of a deep learning retinal image analysis system for diagnosing plus disease in retinopathy of prematurity (ROP). Here we assess the screening potential of this scoring system by determining its ability to detect all components of ROP diagnosis.MethodsClinical examination and fundus photography were performed at seven participating centres. A deep learning system was trained to detect plus disease, generating a quantitative assessment of retinal vascular abnormality (the i-ROP plus score) on a 1–9 scale. Overall ROP disease category was established using a consensus reference standard diagnosis combining clinical and image-based diagnosis. Experts then ranked ordered a second data set of 100 posterior images according to overall ROP severity.Results4861 examinations from 870 infants were analysed. 155 examinations (3%) had a reference standard diagnosis of type 1 ROP. The i-ROP deep learning (DL) vascular severity score had an area under the receiver operating curve of 0.960 for detecting type 1 ROP. Establishing a threshold i-ROP DL score of 3 conferred 94% sensitivity, 79% specificity, 13% positive predictive value and 99.7% negative predictive value for type 1 ROP. There was strong correlation between expert rank ordering of overall ROP severity and the i-ROP DL vascular severity score (Spearman correlation coefficient=0.93; pConclusionThe i-ROP DL system accurately identifies diagnostic categories and overall disease severity in an automated fashion, after being trained only on posterior pole vascular morphology. These data provide proof of concept that a deep learning screening platform could improve objectivity of ROP diagnosis and accessibility of screening.

Retinopathy of prematurity (ROP) is a disease that affects premature infants, where abnormal growth of the retinal blood vessels can lead to blindness unless treated accordingly. Infants considered at risk of severe ROP are monitored for symptoms of plus disease, characterized by arterial tortuosity and venous dilation at the posterior pole, with a standard photographic definition. Disagreement among ROP experts in diagnosing plus disease has driven the development of computer-based methods that classify images based on hand-crafted features extracted from the vasculature. However, most of these approaches are semi-automated, which are time-consuming and subject to variability. In contrast, deep learning is a fully automated approach that has shown great promise in a wide variety of domains, including medical genetics, informatics and imaging. Convolutional neural networks (CNNs) are deep networks which learn rich representations of disease features that are highly robust to variations in acquisition and image quality. In this study, we utilized a U-Net architecture to perform vessel segmentation and then a GoogLeNet to perform disease classification. The classifier was trained on 3,000 retinal images and validated on an independent test set of patients with different observed progressions and treatments. We show that our fully automated algorithm can be used to monitor the progression of plus disease over multiple patient visits with results that are consistent with the experts’ consensus diagnosis. Future work will aim to further validate the method on larger cohorts of patients to assess its applicability within the clinic as a treatment monitoring tool.

Many visual effects are believed to be processed at several functional and anatomical levels of cortical processing. Determining if and how the levels contribute differentially to these effects is a leading problem in visual perception and visual neuroscience. We review and analyze a combination of extant psychophysical findings in the context of neurophysiological and brain-imaging results. Specifically using findings relating to visual illusions, crowding, and masking as exemplary cases, we develop a theoretical rationale for showing how relative levels of cortical processing contributing to these effects can already be deduced from the psychophysically determined functions relating respectively the illusory, crowding and masking strengths to the contrast of the illusion inducers, of the flankers producing the crowding, and of the mask. The wider implications of this rationale show how it can help to settle or clarify theoretical and interpretive inconsistencies and how it can further psychophysical, brain-recording and brain-imaging research geared to explore the relative functional and cortical levels at which conscious and unconscious processing of visual information occur. Our approach also allows us to make some specific predictions for future studies, whose results will provide empirical tests of its validity.

High-throughput phenotyping is a cornerstone of numerous functional genomics projects. In recent years, imaging screens have become increasingly important in understanding gene–phenotype relationships in studies of cells, tissues and whole organisms. Three-dimensional (3D) imaging has risen to prominence in the field of developmental biology for its ability to capture whole embryo morphology and gene expression, as exemplified by the International Mouse Phenotyping Consortium (IMPC). Large volumes of image data are being acquired by multiple institutions around the world that encompass a range of modalities, proprietary software and metadata. To facilitate robust downstream analysis, images and metadata must be standardized to account for these differences. As an open scientific enterprise, making the data readily accessible is essential so that members of biomedical and clinical research communities can study the images for themselves without the need for highly specialized software or technical expertise. In this article, we present a platform of software tools that facilitate the upload, analysis and dissemination of 3D images for the IMPC. Over 750 reconstructions from 80 embryonic lethal and subviable lines have been captured to date, all of which are openly accessible at mousephenotype.org. Although designed for the IMPC, all software is available under an open-source licence for others to use and develop further. Ongoing developments aim to increase throughput and improve the analysis and dissemination of image data. Furthermore, we aim to ensure that images are searchable so that users can locate relevant images associated with genes, phenotypes or human diseases of interest.

PURPOSE Measuring treatment response is vital for assessing efficacy of treatment regimen for patients with brain metastases (BM). Unfortunately, manual delineation of all lesions on MRI across time-points is prohibitively time-consuming, making it infeasible to track individual lesion growth/shrinkage rates as part of the clinical workflow. To overcome this challenge, we propose a deep learning approach to segment all BM, and furthermore, show that certain brain regions are more prone to high-growth rate lesions. METHODS 163 longitudinal MRIs from 77 patients with MPRAGE-post contrast imaging protocol were prospectively obtained from Massachusetts General Hospital (MGH). An expert neuro-oncologist provided ground truth segmentations for all patients. A 3D U-Net architecture was trained to automatically segment BM; training was stopped when validation set Dice score plateaued to prevent overfitting. To enable lesion tracking, all time-points per patient were affinely registered to each other. Every lesion was subsequently classified based on its growth rate (responder: overall lesion shrinkage; inconclusive: 0% to 40% lesion growth; non-responder: more than 40% lesion growth). Characterization of global lesion growth rate patterns was accomplished by affinely registering all time-points to the MNI brain atlas. Segmented lesions were projected onto the atlas, which was qualitatively analyzed to identify spatial regions composed primarily of one class of lesion. RESULTS For automatic segmentation, we report a mean dice score of 0.778, 0.737, and 0.704 on training, validation, and testing sets respectively. Furthermore, we find that the largest BM with the highest average growth rate (non-responders) tend to be located in the posterior frontal/parietal lobes, while smaller, lower growth rate lesions (responders) tend to be localized in the frontal lobes. The posterior fossa was found to be heterogeneous in lesion size and growth rate. CONCLUSION We developed automatic metastatic lesion tracking over time-points and identified brain regions associated with differing growth rate lesions.

The International Mouse Phenotyping Consortium (IMPC) is providing the world's first functional catalogue of a mammalian genome by characterising a knockout mouse strain for every gene. A robust and highly structured informatics platform has been developed to systematically collate, analyse and disseminate the data produced by the IMPC. As the first phase of the project, in which 5000 new knockout strains are being broadly phenotyped, nears completion, the informatics platform is extending and adapting to support the increasing volume and complexity of the data produced as well as addressing a large volume of users and emerging user groups. An intuitive interface helps researchers explore IMPC data by giving overviews and the ability to find and visualise data that support a phenotype assertion. Dedicated disease pages allow researchers to find new mouse models of human diseases, and novel viewers provide high-resolution images of embryonic and adult dysmorphologies. With each monthly release, the informatics platform will continue to evolve to support the increased data volume and to maintain its position as the primary route of access to IMPC data and as an invaluable resource for clinical and non-clinical researchers.

Approximately one third of all mammalian genes are essential for life. Phenotypes resulting from mouse knockouts of these genes have provided tremendous insight into gene function and congenital disorders. As part of the International Mouse Phenotyping Consortium effort to generate and phenotypically characterize 5000 knockout mouse lines, we have identified 410 lethal genes during the production of the first 1751 unique gene knockouts. Using a standardised phenotyping platform that incorporates high-resolution 3D imaging, we identified novel phenotypes at multiple time points for previously uncharacterized genes and additional phenotypes for genes with previously reported mutant phenotypes. Unexpectedly, our analysis reveals that incomplete penetrance and variable expressivity are common even on a defined genetic background. In addition, we show that human disease genes are enriched for essential genes identified in our screen, thus providing a novel dataset that facilitates prioritization and validation of mutations identified in clinical sequencing efforts.

Rheumatoid arthritis (RA) is an autoimmune disease in which chronic inflammation of the synovial joints can lead to destruction of cartilage and bone. Pre-clinical studies attempt to uncover the underlying causes by emulating the disease in genetically different mouse strains and characterising the nature and severity of bone shape changes as indicators of pathology. This paper presents a fully automated method for obtaining quantitative measurements of bone destruction from volumetric micro-CT images of a mouse hind paw. A statistical model of normal bone morphology derived from a training set of healthy examples serves as a template against which a given pathological sample is compared. Abnormalities in bone shapes are identified as deviations from the model statistics, characterised in terms of type (erosion / formation) and quantified in terms of severity (percentage affected bone area). The colour-coded magnitudes of the deviations superimposed on a three-dimensional rendering of the paw show at a glance the severity of malformations for the individual bones and joints. With quantitative data it is possible to derive population statistics characterising differences in bone malformations for different mouse strains and in different anatomical regions. The method was applied to data acquired from three different mouse strains. The derived quantitative indicators of bone destruction have shown agreement both with the subjective visual scores and with the previous biological findings. This suggests that pathological bone shape changes can be usefully and objectively identified as deviations from the model statistics.