Search

A subject’s head position in magnetic resonance imaging (MRI) scanners can vary significantly with the imaging environment and disease status. This variation is known to influence the accuracy of skull stripping (SS), a method to extract the brain region from the whole head image, which is an essential initial step to attain high performance in various neuroimaging applications. However, existing SS methods have failed to accommodate this wide range of variation. To achieve accurate, consistent, and fast SS, we introduce a novel two-stage methodology that we call posture correction skull stripping (PCSS): the first involves adjusting the subject’s head angle and position, and the second involves the actual SS to generate the brain mask. PCSS also incorporates various machine learning techniques, such as a weighted loss function, adversarial training from generative adversarial networks, and ensemble methods. Thorough evaluations conducted on five publicly accessible datasets show that the PCSS method outperforms current state-of-the-art techniques in SS performance, achieving an average increase of 1.38 points on the Dice score and demonstrating the contributions of each PCSS component technique.

There is increasing interest in the use of multimodal data in various web applications, such as digital advertising and e-commerce. Typical methods for extracting important information from multimodal data rely on a mid-fusion architecture that combines the feature representations from multiple encoders. However, as the number of modalities increases, several potential problems with the mid-fusion model structure arise, such as an increase in the dimensionality of the concatenated multimodal features and missing modalities. To address these problems, we propose a new concept that considers multimodal inputs as a set of sequences, namely, deep multimodal sequence sets (DM2S2). Our set-aware concept consists of three components that capture the relationships among multiple modalities: (a) a BERT-based encoder to handle the inter- and intra-order of elements in the sequences, (b) intra-modality residual attention (IntraMRA) to capture the importance of the elements in a modality, and (c) inter-modality residual attention (InterMRA) to enhance the importance of elements with modality-level granularity further. Our concept exhibits performance that is comparable to or better than the previous set-aware models. Furthermore, we demonstrate that the visualization of the learned InterMRA and IntraMRA weights can provide an interpretation of the prediction results.

Although attention mechanisms have been applied to a variety of deep learning models and have been shown to improve the prediction performance, it has been reported to be vulnerable to perturbations to the mechanism. To overcome the vulnerability to perturbations in the mechanism, we are inspired by adversarial training (AT), which is a powerful regularization technique for enhancing the robustness of the models. In this paper, we propose a general training technique for natural language processing tasks, including AT for attention (Attention AT) and more interpretable AT for attention (Attention iAT). The proposed techniques improved the prediction performance and the model interpretability by exploiting the mechanisms with AT. In particular, Attention iAT boosts those advantages by introducing adversarial perturbation, which enhances the difference in the attention of the sentences. Evaluation experiments with ten open datasets revealed that AT for attention mechanisms, especially Attention iAT, demonstrated (1) the best performance in nine out of ten tasks and (2) more interpretable attention (i.e., the resulting attention correlated more strongly with gradient-based word importance) for all tasks. Additionally, the proposed techniques are (3) much less dependent on perturbation size in AT.

To build a robust and practical content-based image retrieval (CBIR) system applicable to clinical brain MRI databases, we propose a new framework, disease-oriented image embedding with pseudo-scanner standardization (DI-PSS). It consists of two core techniques: data harmonization to absorb differences caused by different scanning environments and an algorithm to generate low-dimensional embeddings suitable for disease classification. Until now, there have been very few studies aimed at CBIR of brain MRI. Even in the harmonization of scanners, which is an important prerequisite technique for CBIR, only a limited number of studies have been conducted on T1-weighted MRI, which has collected a vast amount of clinical data. Recently proposed methods need to correctly estimate the domain (i.e., dataset, scanner) of each data in advance to remove environment-dependent information from low-dimensional embedding, which is not an easy task. With DI-PSS, each brain image is pseudo-transformed into a brain image taken with a given reference scanner. Then, 3D convolutional autoencoders (3D-CAE) trained with deep metric learning generate low-dimensional embeddings that better reflect the characteristics of the disease. In this study, DI-PSS reduced the variability of distance in low-dimensional embedding between Alzheimer’s disease (AD) and clinically normal (CN) patients, caused by differences in scanners and datasets, by 15.8-22.6% and 18.0-29.9%, respectively, compared to the baseline. This improved the ability of spectral clustering to classify AD and CN by 6.2% in average accuracy and 10.7% in macro-F1. Our method has the advantage of not requiring difficult domain prediction tasks in advance, and can effectively utilize the big data of T1-weighted MR images. Given the potential of the DI-PSS for harmonizing images scanned by MRI scanners that were not used to scan the training data, it is well suited for application to a large number of legacy MRIs captured in heterogeneous environments.

Discontinuing ad creatives at an appropriate time is one of the most important ad operations that can have a significant impact on sales. Such operational support for ineffective ads has been less explored than that for effective ads. After pre-analyzing 1,000,000 real-world ad creatives, we found that there are two types of discontinuation: short-term (i.e., cut-out) and long-term (i.e., wear-out). In this paper, we propose a practical prediction framework for the discontinuation of ad creatives with a hazard function-based loss function inspired by survival analysis. Our framework predicts the discontinuations with a multi-modal deep neural network that takes as input the ad creative (e.g., text, categorical, image, numerical features). To improve the prediction performance for the two different types of discontinuations and for the ad creatives that contribute to sales, we introduce two new techniques: (1) a two-term estimation technique with multi-task learning and (2) a click-through rate-weighting technique for the loss function. We evaluated our framework using the large-scale ad creative dataset, including 10 billion scale impressions. In terms of the concordance index (short: 0.896, long: 0.939, and overall: 0.792), our framework achieved significantly better performance than the conventional method (0.531). Additionally, we confirmed that our framework (i) demonstrated the same degree of discontinuation effect as manual operations for short-term cases, and (ii) accurately predicted the ad discontinuation order, which is important for long-running ad creatives for long-term cases.

The objective of this study was to evaluate the relation between age and dermatoscopic features of acral nevi. We evaluated 159 dermatoscopic images of melanocytic nevi from 146 individuals filed at the Dermatoscopy Outpatient Clinic of Tokyo Women’s Medical University Medical Center East between April 2006 and March 2009. All images of melanocytic lesions on acral volar skin that showed a clear-cut dermatoscopic pattern of an acral nevus at the time of initial observation were included. The dermatoscopic patterns of all images were retrospectively examined in a blinded fashion according to the standard dermatoscopic classification criteria for acral melanocytic nevi. Images were classified using 15 structural variants of the parallel furrow pattern. These variants were then re-classified into two groups; the “single” line group and “double” line group. Patients of the double line group (age, 25.5 years) were significantly younger than those of the single line group (32.4 years). There was no significant difference in the age-related predominance between the solid line patterns and dotted line patterns. There was a significant age difference between patients with nevi showing the crista dotted pattern (mean age 24.9 years) and patients with nevi without the crista dotted pattern (mean age 34.6 years). We conclude that the double line variant of the parallel furrow pattern and crista dotted pattern, which probably correspond to the congenital type acral nevus, tend to be more common in young patients.

Ductal carcinoma in situ (DCIS) is a pre-invasive carcinoma of the breast that exhibits several distinct morphologies but the link between morphology and patient outcome is not clear. We hypothesize that different mechanisms of growth may still result in similar 2D morphologies, which may look different in 3D. To elucidate the connection between growth and 3D morphology, we reconstruct the 3D architecture of cribriform DCIS from resected patient material. We produce a fully automated algorithm that aligns, segments, and reconstructs 3D architectures from microscopy images of 2D serial sections from human specimens. The alignment algorithm is based on normalized cross correlation, the segmentation algorithm uses histogram equilization, Otsu's thresholding, and morphology techniques to segment the duct and cribra. The reconstruction method combines these images in 3D. We show that two distinct 3D architectures are indeed found in samples whose 2D histological sections are similarly identified as cribriform DCIS. These differences in architecture support the hypothesis that luminal spaces may form due to different mechanisms, either isolated cell death or merging fronds, leading to the different architectures. We find that out of 15 samples, 6 were found to have 'bubble-like' cribra, 6 were found to have 'tube-like' criba and 3 were 'unknown.' We propose that the 3D architectures found, 'bubbles' and 'tubes', account for some of the heterogeneity of the disease and may be prognostic indicators of different patient outcomes.