Search

Objectives: Despite nearly universal prenatal ultrasound screening programs, congenital heart defects (CHD) are still missed, which may result in severe morbidity or even death. Deep machine learning (DL) can automate image recognition from ultrasound. The main aim of this study was to assess the performance of a previously developed DL model, trained on images from a tertiary center, using fetal ultrasound images obtained during the second‐trimester standard anomaly scan in a low‐risk population. A secondary aim was to compare initial screening diagnosis, which made use of live imaging at the point‐of‐care, with diagnosis by clinicians evaluating only stored images. Methods: All pregnancies with isolated severe CHD in the Northwestern region of The Netherlands between 2015 and 2016 with available stored images were evaluated, as well as a sample of normal fetuses' examinations from the same region and time period. We compared the accuracy of the initial clinical diagnosis (made in real time with access to live imaging) with that of the model (which had only stored imaging available) and with the performance of three blinded human experts who had access only to the stored images (like the model). We analyzed performance according to ultrasound study characteristics, such as duration and quality (scored independently by investigators), number of stored images and availability of screening views. Results: A total of 42 normal fetuses and 66 cases of isolated CHD at birth were analyzed. Of the abnormal cases, 31 were missed and 35 were detected at the time of the clinical anatomy scan (sensitivity, 53%). Model sensitivity and specificity were 91% and 78%, respectively. Blinded human experts (n = 3) achieved mean ± SD sensitivity and specificity of 55 ± 10% (range, 47–67%) and 71 ± 13% (range, 57–83%), respectively. There was a statistically significant difference in model correctness according to expert‐graded image quality (P = 0.03). The abnormal cases included 19 lesions that the model had not encountered during its training; the model's performance in these cases (16/19 correct) was not statistically significantly different from that for previously encountered lesions (P = 0.41). Conclusions: A previously trained DL algorithm had higher sensitivity than initial clinical assessment in detecting CHD in a cohort in which over 50% of CHD cases were initially missed clinically. Notably, the DL algorithm performed well on community‐acquired images in a low‐risk population, including lesions to which it had not been exposed previously. Furthermore, when both the model and blinded human experts had access to only stored images and not the full range of images available to a clinician during a live scan, the model outperformed the human experts. Together, these findings support the proposition that use of DL models can improve prenatal detection of CHD. © 2023 International Society of Ultrasound in Obstetrics and Gynecology. [ABSTRACT FROM AUTHOR]

With significant advances in vision and natural language processing, the generation of image captions becomes a need. Mathews, Xie, and He extended a new model to generate styled captions by separating semantics and style. In continuation of their work, here, a new captioning model is developed, including an image encoder to extract the features, a mixture of recurrent networks to embed the set of extracted features to a group of words, and a sentence generator that combines the obtained words as a stylized sentence. This Mixture of Recurrent Experts (MoRE) system uses a new training algorithm that derives singular value decomposition from weighting matrices of Recurrent Neural Networks (RNNs) to increase the diversity of captions. Each decomposition step depends on a distinctive factor based on the number of RNNs in MoRE. The used sentence generator gives a stylized language corpus without paired images. Besides, the styled and diverse captions are extracted without training on a densely labeled or styled dataset. MoRE on the COCO dataset generated diverse and stylized image captions without the necessity of extra‐labeling and improved descriptions in terms of content accuracy. [ABSTRACT FROM AUTHOR]

Purpose: Deep neural networks (DNNs) have been widely applied in medical image classification, benefiting from its powerful mapping capability among medical images. However, these existing deep learning‐based methods depend on an enormous amount of carefully labeled images. Meanwhile, noise is inevitably introduced in the labeling process, degrading the performance of models. Hence, it is significant to devise robust training strategies to mitigate label noise in the medical image classification tasks. Methods: In this work, we propose a novel Bayesian statistics‐guided label refurbishment mechanism (BLRM) for DNNs to prevent overfitting noisy images. BLRM utilizes maximum a posteriori probability in the Bayesian statistics and the exponentially time‐weighted technique to selectively correct the labels of noisy images. The training images are purified gradually with the training epochs when BLRM is activated, further improving classification performance. Results: Comprehensive experiments on both synthetic noisy images (public OCT & Messidor datasets) and real‐world noisy images (ANIMAL‐10N) demonstrate that BLRM refurbishes the noisy labels selectively, curbing the adverse effects of noisy data. Also, the anti‐noise BLRMs integrated with DNNs are effective at different noise ratio and are independent of backbone DNN architectures. In addition, BLRM is superior to state‐of‐the‐art comparative methods of anti‐noise. Conclusions: These investigations indicate that the proposed BLRM is well capable of mitigating label noise in medical image classification tasks. [ABSTRACT FROM AUTHOR]

Image classification has been widely adopted by current social media applications. Compared with fully supervised classification, semisupervised classification attracts more attention because it is commonly observed that category labels are only available for a small portion of images while most images on social media platforms do not have labels. To this end, we propose a two‐stage semisupervised learning framework. In the first stage, we train two Self‐supervised Models (SSMs). One model is initialized by predicting the rotation angles of pretransformed training images and then further trained by the labeled images. The other model is initialized by making consistent predictions for the transformed images in color, shape, and quality from the same sample image, and then further trained by the labeled images. In the second stage, we fuse the two SSMs through deep mutual learning, which enhances each of the two SSMs with the complementary information provided by the other such that the correct prediction could be shared. Experimental results on CIFAR and Caltech‐256 data sets demonstrate the effect of the proposed framework. [ABSTRACT FROM AUTHOR]

Purpose: This study developed and evaluated a fully convolutional network (FCN) for pediatric CT organ segmentation and investigated the generalizability of the FCN across image heterogeneities such as CT scanner model protocols and patient age. We also evaluated the autosegmentation models as part of a software tool for patient‐specific CT dose estimation. Methods: A collection of 359 pediatric CT datasets with expert organ contours were used for model development and evaluation. Autosegmentation models were trained for each organ using a modified FCN 3D V‐Net. An independent test set of 60 patients was withheld for testing. To evaluate the impact of CT scanner model protocol and patient age heterogeneities, separate models were trained using a subset of scanner model protocols and pediatric age groups. Train and test sets were split to answer questions about the generalizability of pediatric FCN autosegmentation models to unseen age groups and scanner model protocols, as well as the merit of scanner model protocol or age‐group‐specific models. Finally, the organ contours resulting from the autosegmentation models were applied to patient‐specific dose maps to evaluate the impact of segmentation errors on organ dose estimation. Results: Results demonstrate that the autosegmentation models generalize to CT scanner acquisition and reconstruction methods which were not present in the training dataset. While models are not equally generalizable across age groups, age‐group‐specific models do not hold any advantage over combining heterogeneous age groups into a single training set. Dice similarity coefficient (DSC) and mean surface distance results are presented for 19 organ structures, for example, median DSC of 0.52 (duodenum), 0.74 (pancreas), 0.92 (stomach), and 0.96 (heart). The FCN models achieve a mean dose error within 5% of expert segmentations for all 19 organs except for the spinal canal, where the mean error was 6.31%. Conclusions: Overall, these results are promising for the adoption of FCN autosegmentation models for pediatric CT, including applications for patient‐specific CT dose estimation. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) that predict mutational status from H&E slides of cancers can enable inexpensive and timely precision oncology. Although expert knowledge is reliable for annotating regions informative of malignancy and other known histologic patterns (strong supervision), it is unreliable for identifying regions informative of mutational status. This poses a serious impediment to obtaining higher prognostic accuracy and discovering new knowledge of pathobiology. We used a weakly supervised learning technique to train a DNN to predict BRAF V600E mutational status, determined using DNA testing, in H&E‐stained images of thyroid cancer tissue without regional annotations. Our discovery cohort was a tissue microarray of only 85 patients from a single hospital. On a large independent external cohort of 444 patients from other hospitals, the trained model gave an area under the receiver operating characteristic curve of 0.98 (95% CI 0.97–1.00), which is much higher than the previously reported results for detecting any mutation using H&E by DNNs trained using strong supervision. We also developed a visualization technique that can automatically highlight regions the DNN found most informative for predicting mutational status. Our visualization is spatially granular and highly specific in highlighting strong negative and positive regions and moves us toward explainable artificial intelligence. Using t‐tests, we confirmed that the proportions of follicular or papillary histology and oncocytic cytology, as noted for each patient by a pathologist who was blinded to the mutational status, were significantly different between mutated and wildtype patients. However, based solely on these features noted by the pathologist, a logistic regression classifier gave an average area under the receiver operating characteristic curve of 0.78 in five‐fold cross‐validation, which is much lower than that obtained using the DNN. These results highlight the potential of weakly supervised learning for training DNN models for problems where the informative visual patterns and their locations are not known a priori. © 2021 The Pathological Society of Great Britain and Ireland. Published by John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

The combination of computer vision and natural language processing is still a very challenging issue. In contrast to previous models focusing on generating only a single sentence for a video, we think that describing a longer video is an important application. In this paper, we propose a video scenario description system that considers video genres to generate multiple sentences. First, the semantics and genres of videos are analyzed. Next, video descriptions are also analyzed. Then, relevant semantic features are selected and translated into the corresponding video descriptions through deep learning. In the experiments, we compare the generated video descriptions based on four evaluation metrics. The results reveal our method is comparable with the state‐of‐the‐art methods. [ABSTRACT FROM AUTHOR]

Multivariate time series (MTS) prediction aims at predicting future time series by extracting multiple forms of dependencies of past time series. Traditional prediction methods and deep learning‐based prediction methods focus on extracting the dynamic relationships of certain aspects of MTS, especially the temporal characteristics, often neglecting the spatial and temporal dynamic correlations of MTS. Inspired by convolution neural network (CNN) and attention mechanism, this paper proposes a convolution LSTM network model based on MTS prediction with two‐stage attention. Specifically, we first propose a new MTS preprocessing method to perform convolution operations better. Then convolution layer extracts spatial correlation of MTS and LSTM model extracts temporal correlation. It is worth mentioning that the combination of attention mechanism and LSTM can effectively solve the problem of insufficient time dependency in MTS prediction. In addition, dual‐stage attention mechanism can effectively eliminate irrelevant information, select the relevant exogenous sequence, give it higher weight, and increase the past value of the target sequence to further eliminate irrelevant information. Finally, the MTS spatio‐temporal correlation is extracted to improve the prediction accuracy, and the model is interpreted. Experimental results show that the model has broad application prospects. Experiments based on typical datasets of finance, environment, and energy determine the optimal window size and hidden size of the prediction, and demonstrate that the model achieves the state‐of‐the‐art effect compared to the other four deep learning models. On top of that, the model is not only suitable for single‐step prediction of MTS, but also suitable for multistep prediction of time step in a certain range. [ABSTRACT FROM AUTHOR]

Purpose: To predict the spatial and temporal trajectories of lung tumor during radiotherapy monitored under a longitudinal magnetic resonance imaging (MRI) study via a deep learning algorithm for facilitating adaptive radiotherapy (ART). Methods: We monitored 10 lung cancer patients by acquiring weekly MRI‐T2w scans over a course of radiotherapy. Under an ART workflow, we developed a predictive neural network (P‐net) to predict the spatial distributions of tumors in the coming weeks utilizing images acquired earlier in the course. The three‐step P‐net consisted of a convolutional neural network to extract relevant features of the tumor and its environment, followed by a recurrence neural network constructed with gated recurrent units to analyze trajectories of tumor evolution in response to radiotherapy, and finally an attention model to weight the importance of weekly observations and produce the predictions. The performance of P‐net was measured with Dice and root mean square surface distance (RMSSD) between the algorithm‐predicted and experts‐contoured tumors under a leave‐one‐out scheme. Results: Tumor shrinkage was 60% ± 27% (mean ± standard deviation) by the end of radiotherapy across nine patients. Using images from the first three weeks, P‐net predicted tumors on future weeks (4, 5, 6) with a Dice and RMSSD of (0.78 ± 0.22, 0.69 ± 0.24, 0.69 ± 0.26), and (2.1 ± 1.1 mm, 2.3 ± 0.8 mm, 2.6 ± 1.4 mm), respectively. Conclusion: The proposed deep learning algorithm can capture and predict spatial and temporal patterns of tumor regression in a longitudinal imaging study. It closely follows the clinical workflow, and could facilitate the decision‐making of ART. A prospective study including more patients is warranted. [ABSTRACT FROM AUTHOR]

Sentiment analysis (also known as opinion mining) is an active research area in natural language processing. It aims at identifying, extracting and organizing sentiments from user generated texts in social networks, blogs or product reviews. A lot of studies in literature exploit machine learning approaches to solve sentiment analysis tasks from different perspectives in the past 15 years. Since the performance of a machine learner heavily depends on the choices of data representation, many studies devote to building powerful feature extractor with domain expert and careful engineering. Recently, deep learning approaches emerge as powerful computational models that discover intricate semantic representations of texts automatically from data without feature engineering. These approaches have improved the state-of-the-art in many sentiment analysis tasks including sentiment classification of sentences/documents, sentiment extraction and sentiment lexicon learning. In this paper, we provide an overview of the successful deep learning approaches for sentiment analysis tasks, lay out the remaining challenges and provide some suggestions to address these challenges. WIREs Data Mining Knowl Discov 2015, 5:292-303. doi: 10.1002/widm.1171 For further resources related to this article, please visit the . [ABSTRACT FROM AUTHOR]