Search

Histopathological Whole Slide Imaging (WSI) has become a standard in the detection of breast cancer. Automated image analysis methods attempt to reduce the workload from the clinicians and Convolutional Neural Networks (CNNs) are a popular choice for this purpose. However, size of a WSI image typically is approximately $40,000\times 40.000$ pixels (can reach up to $100,000\times 100.000$ pixels). CNNs cannot handle such large images. Moreover, downscaling a WSI image causes degradation of small-scale visual information. Hence, a large number of small patches (containing critical visual information) from a WSI image are extracted by a trained pathologist and are used for training. However, it requires massive amounts of time to precisely search and label appropriate class-representative patches. To address this issue, a Deep Multiple Instance Learning (MIL) based CNN framework has been introduced in this paper. In the proposed framework every slide is represented as a bag of extracted patches. Only the bag label is used for training, thus eliminating the requirement to provide patchwise labels. The patches inherit the label of the bag containing them. A WSI image (i.e. a bag) is labeled benign if all its patches are benign and labeled malignant even if a single patch contains malignant cells. Learning can be carried out at the bag level even with noisy patch labels. Performance of this method was evaluated using the BreakHis, IUPHL and UCSB breast cancer datasets where 93.06%, 96.63%, 95.83% accuracy was achieved respectively.

Traditional neuroimage analysis pipelines involve computationally intensive, time-consuming optimization steps, and thus, do not scale well to large cohort studies with thousands or tens of thousands of individuals. In this work we propose a fast and accurate deep learning based neuroimaging pipeline for the automated processing of structural human brain MRI scans, replicating FreeSurfer's anatomical segmentation including surface reconstruction and cortical parcellation. To this end, we introduce an advanced deep learning architecture capable of whole brain segmentation into 95 classes. The network architecture incorporates local and global competition via competitive dense blocks and competitive skip pathways, as well as multi-slice information aggregation that specifically tailor network performance towards accurate segmentation of both cortical and sub-cortical structures. Further, we perform fast cortical surface reconstruction and thickness analysis by introducing a spectral spherical embedding and by directly mapping the cortical labels from the image to the surface. This approach provides a full FreeSurfer alternative for volumetric analysis (in under 1 minute) and surface-based thickness analysis (within only around 1h runtime). For sustainability of this approach we perform extensive validation: we assert high segmentation accuracy on several unseen datasets, measure generalizability and demonstrate increased test-retest reliability, and high sensitivity to group differences in dementia., Comment: Submitted to NeuroImage

Purpose: Development of a fast and fully automated deep learning pipeline (FatSegNet) to accurately identify, segment, and quantify abdominal adipose tissue on Dixon MRI from the Rhineland Study - a large prospective population-based study. Method: FatSegNet is composed of three stages: (i) consistent localization of the abdominal region using two 2D-Competitive Dense Fully Convolutional Networks (CDFNet), (ii) segmentation of adipose tissue on three views by independent CDFNets, and (iii) view aggregation. FatSegNet is trained with 33 manually annotated subjects, and validated by: 1) comparison of segmentation accuracy against a testingset covering a wide range of body mass index (BMI), 2) test-retest reliability, and 3) robustness in a large cohort study. Results: The CDFNet demonstrates increased robustness compared to traditional deep learning networks. FatSegNet dice score outperforms manual raters on the abdominal visceral adipose tissue (VAT, 0.828 vs. 0.788), and produces comparable results on subcutaneous adipose tissue (SAT, 0.973 vs. 0.982). The pipeline has very small test-retest absolute percentage difference and excellent agreement between scan sessions (VAT: APD = 2.957%, ICC=0.998 and SAT: APD= 3.254%, ICC=0.996). Conclusion: FatSegNet can reliably analyze a 3D Dixon MRI in1 min. It generalizes well to different body shapes, sensitively replicates known VAT and SAT volume effects in a large cohort study, and permits localized analysis of fat compartments., Comment: Submitted to Magnetic Resonance in Medicine

Background: Improvements to prognostic models in metastatic castration-resistant prostate cancer have the potential to augment clinical trial design and guide treatment strategies. In partnership with Project Data Sphere, a not-for-profit initiative allowing data from cancer clinical trials to be shared broadly with researchers, we designed an open-data, crowdsourced, DREAM (Dialogue for Reverse Engineering Assessments and Methods) challenge to not only identify a better prognostic model for prediction of survival in patients with metastatic castration-resistant prostate cancer but also engage a community of international data scientists to study this disease. Methods Data from the comparator arms of four phase 3 clinical trials in first-line metastatic castration-resistant prostate cancer were obtained from Project Data Sphere, comprising 476 patients treated with docetaxel and prednisone from the ASCENT2 trial, 526 patients treated with docetaxel, prednisone, and placebo in the MAINSAIL trial, 598 patients treated with docetaxel, prednisone or prednisolone, and placebo in the VENICE trial, and 470 patients treated with docetaxel and placebo in the ENTHUSE 33 trial. Datasets consisting of more than 150 clinical variables were curated centrally, including demographics, laboratory values, medical history, lesion sites, and previous treatments. Data from ASCENT2, MAINSAIL, and VENICE were released publicly to be used as training data to predict the outcome of interestnamely, overall survival. Clinical data were also released for ENTHUSE 33, but data for outcome variables (overall survival and event status) were hidden from the challenge participants so that ENTHUSE 33 could be used for independent validation. Methods were evaluated using the integrated time-dependent area under the curve (iAUC). The reference model, based on eight clinical variables and a penalised Cox proportional-hazards model, was used to compare method performance. Further validation was done using data from a fifth trialENTHUSE M1in which 266 patients with metastatic castration-resistant prostate cancer were treated with placebo alone. Findings 50 independent methods were developed to predict overall survival and were evaluated through the DREAM challenge. The top performer was based on an ensemble of penalised Cox regression models (ePCR), which uniquely identified predictive interaction effects with immune biomarkers and markers of hepatic and renal function. Overall, ePCR outperformed all other methods (iAUC 0·791; Bayes factor >5) and surpassed the reference model (iAUC 0·743; Bayes factor >20). Both the ePCR model and reference models stratified patients in the ENTHUSE 33 trial into high-risk and low-risk groups with significantly different overall survival (ePCR: hazard ratio 3·32, 95% CI 2·394·62, p, European Union within the ERC grant LatentCauses supported the work of C.F and I.K. German Research Foundation (DFG) within the Collaborative Research Centre 1243, subproject A17 awarded to C.F. German Federal Ministry of Education and Research (BMBF) through the Research Consortium e:AtheroMED (Systems medicine of myocardial infarction and stroke) under the auspices of the e:Med Programme (grant # 01ZX1313C) supported the work of D.P.A., P.D., C.F., C.K., I.K., N.K., M.L., H.S. and J.F.S. at the Institute of Computational Biology. NIH Grants RR025747-01, MH086633 and 1UL1TR001111, and NSF Grants SES-1357666, DMS-14-07655 and BCS0826844 supported the work of C.H., J.I., E.L., Y.W., H.Y., H.Z. and J.Z. NSFC Grant Nos. 61332013, 61572139 supported the work of X.L, Y.L, Y.Z., and S.Z. National Natural Science Foundation of China grants [Nos. 61422309, 61379092] was awarded to S.Z. The Patrick C. Walsh Prostate Research Fund and the Johns Hopkins Individualized Health Initiative supported the work of R.Y.C., D.D., Y.D., Z.J., K.R., Z.W. and Y.Z. FCT Ph.D. Grant SFRH/BD/80925/2011 was awarded to S.C. Clinical Persona Inc., East Palo Alto, CA supported the work of L.B. and D.K. The Finnish Cultural Foundation and the Drug Research Doctoral Programme (DRDP) at the University of Turku supported T.D.L. The National Research Foundation Singapore and the Singapore Ministry of Education, under its Research Centres of Excellence initiative, supported the work of J.G. and K.T. A grant from the Russian Science Foundation 14-24-00155 was awarded to M.D.K. A*MIDEX grant (no. ANR-11-IDEX-0001-02) was awarded to P.J.B. NSERC supported the work of R.G. The Israeli Centers of Research Excellence (I-CORE) program (Center No. 4/11) supported the work of E.G. Academy of Finland (grants 292611, 269862, 272437, 279163, 295504), National Cancer Institute (16X064), and Cancer Society of Finland supported the work of T.A. Academy of Finland (grant 268531) supported the work of T.M

International audience; Surgical tool detection is attracting increasing attention from the medical image analysis community. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design efficient solutions for surgical workflow analysis, with potential applications in report generation, surgical training and even real-time decision support. Most existing tool annotation algorithms focus on laparo-scopic surgeries. However, with 19 million interventions per year, the most common surgical procedure in the world is cataract surgery. The CATARACTS challenge was organized in 2017 to evaluate tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. With 14 participating teams, this challenge can be considered a success. As might be expected,-H. Conze et al. / Medical Image Analysis 52 (2019) 24-41 25 the submitted solutions are based on deep learning. This paper thoroughly evaluates these solutions: in particular, the quality of their annotations are compared to that of human interpretations. Next, lessons learnt from the differential analysis of these solutions are discussed. We expect that they will guide the design of efficient surgery monitoring tools in the near future.

Intravascular imaging using ultrasound or optical coherence tomography (OCT) is predominantly used to adjunct clinical information in interventional cardiology. OCT provides high-resolution images for detailed investigation of atherosclerosis-induced thickening of the lumen wall resulting in arterial blockage and triggering acute coronary events. However, the stochastic uncertainty of speckles limits effective visual investigation over large volume of pullback data, and clinicians are challenged by their inability to investigate subtle variations in the lumen topology associated with plaque vulnerability and onset of necrosis. This paper presents a lumen segmentation method using OCT imaging physics-based graph representation of signals and random walks image segmentation approaches. The edge weights in the graph are assigned incorporating OCT signal attenuation physics models. Optical backscattering maxima is tracked along each A-scan of OCT and is subsequently refined using global graylevel statistics and used for initializing seeds for the random walks image segmentation. Accuracy of lumen versus tunica segmentation has been measured on 15 in vitro and 6 in vivo pullbacks, each with 150–200 frames using 1) Cohen's kappa coefficient $(0.9786 \pm 0.0061)$ measured with respect to cardiologist's annotation and 2) divergence of histogram of the segments computed with Kullback–Leibler $(5.17 \pm 2.39)$ and Bhattacharya measures $(0.56 \pm 0.28)$ . High segmentation accuracy and consistency substantiates the characteristics of this method to reliably segment lumen across pullbacks in the presence of vulnerability cues and necrotic pool and has a deterministic finite time-complexity. This paper in general also illustrates the development of methods and framework for tissue classification and segmentation incorporating cues of tissue–energy interaction physics in imaging.

Several diseases of parkinsonian syndromes present similar symptoms at early stage and no objective widely used diagnostic methods have been approved until now. Positron emission tomography (PET) with \(^{18}\)F-FDG was shown to be able to assess early neuronal dysfunction of synucleinopathies and tauopathies. Tensor factorization (TF) based approaches have been applied to identify characteristic metabolic patterns for differential diagnosis. However, these conventional dimension-reduction strategies assume linear or multi-linear relationships inside data, and are therefore insufficient to distinguish nonlinear metabolic differences between various parkinsonian syndromes. In this paper, we propose a Deep Projection Neural Network (DPNN) to identify characteristic metabolic pattern for early differential diagnosis of parkinsonian syndromes. We draw our inspiration from the existing TF methods. The network consists of a (i) compression part: which uses a deep network to learn optimal 2D projections of 3D scans, and a (ii) classification part: which maps the 2D projections to labels. The compression part can be pre-trained using surplus unlabelled datasets. Also, as the classification part operates on these 2D projections, it can be trained end-to-end effectively with limited labelled data, in contrast to 3D approaches. We show that DPNN is more effective in comparison to existing state-of-the-art and plausible baselines.

Background Evaluation of molecular pathology markers using a computer-aided quantitative assessment framework would help to assess the altered states of cellular proliferation, hypoxia, and neoangiogenesis in oral submucous fibrosis and could improve diagnostic interpretation in gauging its malignant potentiality. Methods Immunohistochemical (IHC) expression of c-Myc, hypoxia-inducible factor-1-alpha (HIF-1α), vascular endothelial growth factor (VEGF), VEGFRII, and CD105 were evaluated in 58 biopsies of oral submucous fibrosis using computer-aided quantification. After digital stain separation of original chromogenic IHC images, quantification of the diaminobenzidine (DAB) reaction pattern was performed based on intensity and extent of cytoplasmic, nuclear, and stromal expression. Results Assessment of molecular expression proposed that c-Myc and HIF-1α may be used as strong screening markers, VEGF for risk-stratification and VEGFRII and CD105 for prognosis of precancer into oral cancer. Conclusion Our analysis indicated that the proposed method can help in establishing IHC as an effective quantitative immunoassay for molecular pathology and alleviate diagnostic ambiguities in the clinical decision process. © 2015 Wiley Periodicals, Inc. Head Neck, 2015

Intravascular ultrasound (IVUS) is a real-time cross-sectional imaging modality deployed in interventional cardiology for assessment of the extent of atherosclerosis. Visual reading of IVUS pull-backs is subject to inter- and intra-observer variability in reporting of vulnerable plaques causing myocardial infraction. In vivo IVUS tissue characterization (TC) aims at augmenting information about the constituent tissues beyond features visible in the log-compressed B-Mode scan by effectively leveraging characteristic ultrasonic backscattered signals acquired during live intervention. The co-located heterogeneity of biological tissues constituting the plaque, the presence of flowing blood and vessel dynamics openly challenge in vivo TC. As a solution, we introduce a framework that first uses a decision forest based classifier that learns to perform TC using tissue specific ultrasonic statistical physics and signal confidence features, from labeled data acquired under controlled in vitro conditions. Next, we adapt this in vitro trained classifier to work under in vivo settings through a novel error-correcting hierarchical transfer relaxation scheme for domain adaptation with few labeled samples. This effectively compensates for the shift in statistical features between in vitro and in vivo settings owed to the presence of flowing blood and vessel dynamic movements. Experiments reveal the ability of the framework to estimate constituents of the plaque reliably under both in vitro and in vivo settings. This framework can be leveraged for promising clinically applications requiring TC and to perform domain adaptation in the presence of few labeled samples.

Ensemble methods have been successfully applied in a wide range of scenarios, including survival analysis. However, most ensemble models for survival analysis consist of models that all optimize the same loss function and do not fully utilize the diversity in available models. We propose heterogeneous survival ensembles that combine several survival models, each optimizing a different loss during training. We evaluated our proposed technique in the context of the Prostate Cancer DREAM Challenge, where the objective was to predict survival of patients with metastatic, castrate-resistant prostate cancer from patient records of four phase III clinical trials. Results demonstrate that a diverse set of survival models were preferred over a single model and that our heterogeneous ensemble of survival models outperformed all competing methods with respect to predicting the exact time of death in the Prostate Cancer DREAM Challenge.

Optical coherence tomography (OCT) is used for non-invasive diagnosis of diabetic macular edema assessing the retinal layers. In this paper, we propose a new fully convolutional deep architecture, termed ReLayNet, for end-to-end segmentation of retinal layers and fluid masses in eye OCT scans. ReLayNet uses a contracting path of convolutional blocks (encoders) to learn a hierarchy of contextual features, followed by an expansive path of convolutional blocks (decoders) for semantic segmentation. ReLayNet is trained to optimize a joint loss function comprising of weighted logistic regression and Dice overlap loss. The framework is validated on a publicly available benchmark dataset with comparisons against five state-of-the-art segmentation methods including two deep learning based approaches to substantiate its effectiveness., Comment: Accepted for Publication at Biomedical Optics Express

Increasing incidents of diabetes (mellitus) induced non-healing lower extremity wounds and disease associated amputations have raised significant concerns related to quality of life of afflicted patients. High glucose level in diabetic wounds inhibits the transactivation of angiogenesis related molecules resulting delayed healing progression. Present study investigates the impact of a natural medicine like honey in angiogenesis of non-healing diabetic lower limb wounds. Quantitative assessment of different vessel parameters was performed on in vitro CAM model for validation of angiogenic potential of honey. Further the upregulation of angiogenesis related prime molecular markers like HIF-1α, VEGFA, VEGFR2 is under the therapeutic intervention of honey indicated improved angiogenesis which in turn promote the healing rate. These results may facilitate in determining the healing impact of this natural product in treatment of diabetic wounds and it may also help in developing alternative cost effective therapeutic modality.

Interventional cardiologists use intravascular imaging techniques like optical coherence tomography (OCT) as adjunct to angiography for detailed diagnosis of atherosclerosis. Each tissue type is associated with characteristic speckle intensity distribution, which forms the basis for tissue characterization (TC). Classical approaches follow statistical machine learning using apriori assumed speckle models, and are challenged by inability to discriminate high tissue heterogeneity. As a first of its kind approach, we solve this problem in absence of a well studied distribution, by learning the multiscale statistical distribution model of the data using our proposed distribution preserving (DP) autoencoder (AE) based neural network (NN). The learning rule introduces a scale importance parameter associated with error backpropagation. We have evaluated performance of DPAE vs. prior-art and AE (with L2 norm and cross-entropy cost function) to obtain LogLoss of 0.16, 0.28, 0.22, 0.53 respectively, and 93.6% average classification accuracy with DPAE predictions were judged to be clinically acceptable.

Atherosclerosis assessment using the high resolution of intravascular optical coherence tomography (IV-OCT) enables the visualisation of tissue pools which are susceptible to rupture. However, the stochastic nature of OCT speckles leads to subjectivity in interpretation. In this paper, we present a framework for to OCT tissue energy interaction driven marker for identifying susceptible pool of tissues. The local statistics of OCT speckles is coupled with estimate of optical attenuation and signal confidence measures in a multiscale approach. Further, a bag of random forests is trained on a bank of 22 OCT cross-sections where susceptible tissues have been identified by correlating with the histology. Evaluating the performance on 22 other diverse cross-sections substantiates robustness of the framework. It is envisaged that this framework would help in objective assessment of plaque susceptibility and aid in more confident decision making during interventions.

In this work, we cast a multi-modal immunofluorescence to histology registration problem is cast to a mono-modal one by iteratively propagating tissue-specific features from one modality to generate intensities in the other modality via an implicitly learnt random-forest regression framework. The proposed method iterates between modality propagation and image registration in a unified formulation. Evaluations on real and simulated tissue deformations establish superiority of the proposed work over comparative methods in handling highly complex intermodal intensity relationships. This framework will aid in quantitative analysis of the tissue structure, functionality thus providing for co-located molecular validations and bringing in spatial fidelity to histological assessment.

In this paper, we introduce a framework for simulating intravascular ultrasound (IVUS) images and radiofrequency (RF) signals from histology image counterparts. We modeled the wave propagation through the Westervelt equation, which is solved explicitly with a finite differences scheme in polar coordinates by taking into account attenuation and non-linear effects. Our results demonstrate good correlation for textural and spectral information driven from simulated IVUS data in contrast to real data, acquired with single-element mechanically rotating 40 MHz transducer, as ground truth.

Loss of visual acuity on account of retina-related vision impairment can be partly prevented through periodic screening with fundus color imaging. Largescale screening is currently challenged by inability to exhaustively detect fine blood vessels crucial to disease diagnosis. In this work we present a framework for reliable blood vessel detection in fundus color imaging through inductive transfer learning of photon-tissue interaction statistical physics. The source task estimates photon-tissue interaction as a spatially localized Poisson process of photons sensed by the RGB sensor. The target task identifies vascular and non-vascular tissues using knowledge transferred from source task. The source and target domains are retinal images obtained using a color fundus camera with white-light illumination. In experimental evaluation with the DRIVE database, we achieve the objective of vessel detection with max. avg. accuracy of 0.9766 and kappa of 0.8213.

Background: Oral submucous fibrosis (OSF) is a pre-cancerous condition with features of chronic, inflammatory and progressive sub-epithelial fibrotic disorder of the buccal mucosa. In this study, malignant potentiality of OSF has been assessed by quantification of immunohistochemical expression of epithelial prime regulator-p63 molecule in correlation to its malignant (oral squamous cell carcinoma [OSCC] and normal counterpart [normal oral mucosa [NOM]). Attributes of spatial extent and distribution of p63 + expression in the epithelium have been investigated. Further, a correlated assessment of histopathological attributes inferred from H&E staining and their mathematical counterparts (molecular pathology of p63) have been proposed. The suggested analytical framework envisaged standardization of the immunohistochemistry evaluation procedure for the molecular marker, using computer-aided image analysis, toward enhancing its prognostic value. Subjects and Methods: In histopathologically confirmed OSF, OSCC and NOM tissue sections, p63 + nuclei were localized and segmented by identifying regional maxima in plateau-like intensity spatial profiles of nuclei. The clustered nuclei were localized and segmented by identifying concave points in the morphometry and by marker-controlled watersheds. Voronoi tessellations were constructed around nuclei centroids and mean values of spatial-relation metrics such as tessellation area, tessellation perimeter, roundness factor and disorder of the area were extracted. Morphology and extent of expression are characterized by area, diameter, perimeter, compactness, eccentricity and density, fraction of p63 + expression and expression distance of p63 + nuclei. Results: Correlative framework between histopathological features characterizing malignant potentiality and their quantitative p63 counterparts was developed. Statistical analyses of mathematical trends were evaluated between different biologically relevant combinations: (i) NOM to oral submucous fibrosis without dysplasia (OSFWT) (ii) NOM to oral submucous fibrosis with dysplasia (OSFWD) (iii) OSFWT-OSFWD (iv) OSFWD-OSCC. Significant histopathogical correlates and their corroborative mathematical features, inferred from p63 staining, were also investigated into. Conclusion: Quantitative assessment and correlative analysis identified mathematical features related to hyperplasia, cellular stratification, differentiation and maturation, shape and size, nuclear crowding and nucleocytoplasmic ratio. It is envisaged that this approach for analyzing the p63 expression and its distribution pattern may help to establish it as a quantitative bio-marker to predict the malignant potentiality and progression. The proposed work would be a value addition to the gold standard by incorporating an observer-independent framework for the associated molecular pathology.