Search

Most deep learning models for temporal regression directly output the estimation based on single input images, ignoring the relationships between different images. In this paper, we propose deep relation learning for regression, aiming to learn different relations between a pair of input images. Four non-linear relations are considered: “cumulative relation,” “relative relation,” “maximal relation” and “minimal relation.” These four relations are learned simultaneously from one deep neural network which has two parts: feature extraction and relation regression. We use an efficient convolutional neural network to extract deep features from the pair of input images and apply a Transformer for relation learning. The proposed method is evaluated on a merged dataset with 6,049 subjects with ages of 0–97 years using 5-fold cross-validation for the task of brain age estimation. The experimental results have shown that the proposed method achieved a mean absolute error (MAE) of 2.38 years, which is lower than the MAEs of 8 other state-of-the-art algorithms with statistical significance (p<0.05) in paired T-test (two-side). [ABSTRACT FROM AUTHOR]

Text clustering is a critical step in text data analysis and has been extensively studied by the text mining community. Most existing text clustering algorithms are based on the bag-of-words model, which faces the high-dimensional and sparsity problems and ignores text structural and sequence information. Deep learning-based models such as convolutional neural networks and recurrent neural networks regard texts as sequences but lack supervised signals and explainable results. In this paper, we propose a deep feature-based text clustering (DFTC) framework that incorporates pretrained text encoders into text clustering tasks. This model, which is based on sequence representations, breaks the dependency on supervision. The experimental results show that our model outperforms classic text clustering algorithms and the state-of-the-art pretrained language model, i.e., BERT, on almost all the considered datasets. In addition, the explanation of the clustering results is significant for understanding the principles of the deep learning approach. Our proposed clustering framework includes an explanation module that can help users understand the meaning and quality of the clustering results. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) are currently the method of choice both for generative, as well as for discriminative learning in computer vision and machine learning. The success of DCNNs can be attributed to the careful selection of their building blocks (e.g., residual blocks, rectifiers, sophisticated normalization schemes, to mention but a few). In this paper, we propose $\Pi$ Π -Nets, a new class of function approximators based on polynomial expansions. $\Pi$ Π -Nets are polynomial neural networks, i.e., the output is a high-order polynomial of the input. The unknown parameters, which are naturally represented by high-order tensors, are estimated through a collective tensor factorization with factors sharing. We introduce three tensor decompositions that significantly reduce the number of parameters and show how they can be efficiently implemented by hierarchical neural networks. We empirically demonstrate that $\Pi$ Π -Nets are very expressive and they even produce good results without the use of non-linear activation functions in a large battery of tasks and signals, i.e., images, graphs, and audio. When used in conjunction with activation functions, $\Pi$ Π -Nets produce state-of-the-art results in three challenging tasks, i.e., image generation, face verification and 3D mesh representation learning. The source code is available at https://github.com/grigorisg9gr/polynomial_nets. [ABSTRACT FROM AUTHOR]

Using deep neural networks (DNNs) to explore spatial patterns and temporal dynamics of human brain activities has been an important yet challenging problem because the artificial neural networks are hard to be designed manually. There have been several promising deep learning methods, e.g., deep belief network (DBN), convolutional neural network (CNN), and deep sparse recurrent auto-encoder (DSRAE), that can decompose neuroscientific and meaningful spatiotemporal patterns from 4D functional Magnetic Resonance Imaging (fMRI) data. However, those previous studies still depend on hand-crafted neural network architectures and hyperparameters, which are not optimal in various senses. In this paper, we employ the evolutionary algorithms (EA) to optimize the deep neural architecture of DSRAE by minimizing the expected loss of initialized models, named eNAS-DSRAE (evolutionary Neural Architecture Search on Deep Sparse Recurrent Auto-Encoder). Also, validation experiments are designed and performed on the publicly available human connectome project (HCP) 900 datasets, and the results achieved by the optimized eNAS-DSRAE suggested that our framework can successfully identify the spatiotemporal features and perform better than the hand-crafted neural network models. To our best knowledge, the proposed eNAS-DSRAE is not only among the earliest NAS models that can extract connectome-scale meaningful spatiotemporal brain networks from 4D fMRI data, but also is an effective framework to optimize the RNN-based models. [ABSTRACT FROM AUTHOR]

In this paper, we propose an end-to-end deep learning architecture that generates 3D triangular meshes from single color images. Restricted by the nature of prevalent deep learning techniques, the majority of previous works represent 3D shapes in volumes or point clouds. However, it is non-trivial to convert these representations to compact and ready-to-use mesh models. Unlike the existing methods, our network represents 3D shapes in meshes, which are essentially graphs and well suited for graph-based convolutional neural networks. Leveraging perceptual features extracted from an input image, our network produces the correct geometry by progressively deforming an ellipsoid. To make the whole deformation procedure stable, we adopt a coarse-to-fine strategy, and define various mesh/surface related losses to capture properties of various aspects, which benefits producing the visually appealing and physically accurate 3D geometry. In addition, our model by nature can be adapted to objects in specific domains, e.g., human faces, and be easily extended to learn per-vertex properties, e.g., color. Extensive experiments show that our method not only qualitatively produces the mesh model with better details, but also achieves the higher 3D shape estimation accuracy compared against the state-of-the-arts. [ABSTRACT FROM AUTHOR]

It is appealing but challenging to achieve real-time deep neural network (DNN) inference on mobile devices, because even the powerful modern mobile devices are considered as “resource-constrained” when executing large-scale DNNs. It necessitates the sparse model inference via weight pruning, i.e., DNN weight sparsity, and it is desirable to design a new DNN weight sparsity scheme that can facilitate real-time inference on mobile devices while preserving a high sparse model accuracy. This paper designs a novel mobile inference acceleration framework GRIM that is General to both convolutional neural networks (CNNs) and recurrent neural networks (RNNs) and that achieves Real-time execution and high accuracy, leveraging fine-grained structured sparse model Inference and compiler optimizations for Mobiles. We start by proposing a new fine-grained structured sparsity scheme through the Block-based Column-Row (BCR) pruning. Based on this new fine-grained structured sparsity, our GRIM framework consists of two parts: (a) the compiler optimization and code generation for real-time mobile inference; and (b) the BCR pruning optimizations for determining pruning hyperparameters and performing weight pruning. We compare GRIM with Alibaba MNN, TVM, TensorFlow-Lite, a sparse implementation based on CSR, PatDNN, and ESE (a representative FPGA inference acceleration framework for RNNs), and achieve up to $14.08\times$ 14. 08 × speedup. [ABSTRACT FROM AUTHOR]

Handwritten signature verification is a challenging task because signatures of a writer may be skillfully imitated by a forger. As skilled forgeries are generally difficult to acquire for training, in this paper, we propose a deep learning-based dynamic signature verification framework, SynSig2Vec, to address the skilled forgery attack without training with any skilled forgeries. Specifically, SynSig2Vec consists of a novel learning-by-synthesis method for training and a 1D convolutional neural network model, called Sig2Vec, for signature representation extraction. The learning-by-synthesis method first applies the Sigma Lognormal model to synthesize signatures with different distortion levels for genuine template signatures, and then learns to rank these synthesized samples in a learnable representation space based on average precision optimization. The representation space is achieved by the proposed Sig2Vec model, which is designed to extract fixed-length representations from dynamic signatures of arbitrary lengths. Through this training method, the Sig2Vec model can extract extremely effective signature representations for verification. Our SynSig2Vec framework requires only genuine signatures for training, yet achieves state-of-the-art performance on the largest dynamic signature database to date, DeepSignDB, in both skilled forgery and random forgery scenarios. Source codes of SynSig2Vec will be available at https://github.com/LaiSongxuan/SynSig2Vec. [ABSTRACT FROM AUTHOR]

Deep neural networks for medical image reconstruction are traditionally trained using high-quality ground-truth images as training targets. Recent work on Noise2Noise (N2N) has shown the potential of using multiple noisy measurements of the same object as an alternative to having a ground-truth. However, existing N2N-based methods are not suitable for learning from the measurements of an object undergoing nonrigid deformation. This paper addresses this issue by proposing the deformation-compensated learning (DeCoLearn) method for training deep reconstruction networks by compensating for object deformations. A key component of DeCoLearn is a deep registration module, which is jointly trained with the deep reconstruction network without any ground-truth supervision. We validate DeCoLearn on both simulated and experimentally collected magnetic resonance imaging (MRI) data and show that it significantly improves imaging quality. [ABSTRACT FROM AUTHOR]

This paper generalizes the Attention in Attention (AiA) mechanism, in P. Fang et al., 2019 by employing explicit mapping in reproducing kernel Hilbert spaces to generate attention values of the input feature map. The AiA mechanism models the capacity of building inter-dependencies among the local and global features by the interaction of inner and outer attention modules. Besides a vanilla AiA module, termed linear attention with AiA, two non-linear counterparts, namely, second-order polynomial attention and Gaussian attention, are also proposed to utilize the non-linear properties of the input features explicitly, via the second-order polynomial kernel and Gaussian kernel approximation. The deep convolutional neural network, equipped with the proposed AiA blocks, is referred to as Attention in Attention Network (AiA-Net). The AiA-Net learns to extract a discriminative pedestrian representation, which combines complementary person appearance and corresponding part features. Extensive ablation studies verify the effectiveness of the AiA mechanism and the use of non-linear features hidden in the feature map for attention design. Furthermore, our approach outperforms current state-of-the-art by a considerable margin across a number of benchmarks. In addition, state-of-the-art performance is also achieved in the video person retrieval task with the assistance of the proposed AiA blocks. [ABSTRACT FROM AUTHOR]

We observed that remarkable and impressive performance on image-based human pose estimation have been achieved by deep Convolutional Neural Networks (CNN). Nevertheless, directly applying these image-based models on videos is not only computionally intensive, but also may cause jitter and loss. The main reason is that the image-based models purely focus on the local features of individual frames and totally ignore the temporal information among adjacent frames. Some existing methods are proposed to address the temporal coherency issue. However, these methods need to be designed carefully and cannot be combined with existing image-based methods. In this paper, we propose a simple yet effective module to refine the estimated pose by exploiting the temporal coherency among the heatmaps of adjacent frames, which can be easily inserted into image-based networks as a plug-in. We show that the temporal coherency issue among the heatmap frames could be re-formulated as a graph path selection optimization problem. Moreover, to speed up the refinement process, we propose a hierarchical graph optimization to achieve the refinement from coarse to fine. Experimental results on two large-scale video pose estimation benchmarks show that our module can improve the performance with little speed loss when combined with image-based methods as an efficient plug-in. [ABSTRACT FROM AUTHOR]

With the fast development of various positioning techniques such as Global Position System (GPS), mobile devices and remote sensing, spatio-temporal data has become increasingly available nowadays. Mining valuable knowledge from spatio-temporal data is critically important to many real-world applications including human mobility understanding, smart transportation, urban planning, public safety, health care and environmental management. As the number, volume and resolution of spatio-temporal data increase rapidly, traditional data mining methods, especially statistics-based methods for dealing with such data are becoming overwhelmed. Recently deep learning models such as recurrent neural network (RNN) and convolutional neural network (CNN) have achieved remarkable success in many domains due to the powerful ability in automatic feature representation learning, and are also widely applied in various spatio-temporal data mining (STDM) tasks such as predictive learning, anomaly detection and classification. In this paper, we provide a comprehensive review of recent progress in applying deep learning techniques for STDM. We first categorize the spatio-temporal data into five different types, and then briefly introduce the deep learning models that are widely used in STDM. Next, we classify existing literature based on the types of spatio-temporal data, the data mining tasks, and the deep learning models, followed by the applications of deep learning for STDM in different domains including transportation, on-demand service, climate & weather analysis, human mobility, location-based social network, crime analysis, and neuroscience. Finally, we conclude the limitations of current research and point out future research directions. [ABSTRACT FROM AUTHOR]

Due to occlusion among views and temporal inconsistency in depth video, spatio-temporal distortion occurs in 3D synthesized video with depth image-based rendering. In this paper, we propose a deep Convolutional Neural Network (CNN)-based synthesized video denoising algorithm to reduce temporal flicker distortion and improve perceptual quality of 3D synthesized video. First, we analyze the spatio-temporal distortion, and model eliminating spatio-temporal distortion as a perceptual video denoising problem. Then, a deep learning-based synthesized video denoising network is proposed, in which a CNN-friendly spatio-temporal loss function is derived from a synthesized video quality metric and integrated with a single image denoising network architecture. Finally, specific schemes, i.e., specific Synthesized Video Denoising Networks (SynVD-Nets), and a general scheme, i.e., General SynVD-Net (GSynVD-Net), based on existing CNN-based denoising models, are developed to handle synthesized video with different distortion levels more effectively. Experimental results show that the proposed SynVD-Net and GSynVD-Net can outperform deep learning-based counterparts and conventional denoising methods, and significantly enhance perceptual quality of 3D synthesized video. [ABSTRACT FROM AUTHOR]

Monitoring the depth of unconsciousness during anesthesia is beneficial in both clinical settings and neuroscience investigations to understand brain mechanisms. Electroencephalogram (EEG) has been used as an objective means of characterizing brain altered arousal and/or cognition states induced by anesthetics in real-time. Different general anesthetics affect cerebral electrical activities in different ways. However, the performance of conventional machine learning models on EEG data is unsatisfactory due to the low Signal to Noise Ratio (SNR) in the EEG signals, especially in the office-based anesthesia EEG setting. Deep learning models have been used widely in the field of Brain Computer Interface (BCI) to perform classification and pattern recognition tasks due to their capability of good generalization and handling noises. Compared to other BCI applications, where deep learning has demonstrated encouraging results, the deep learning approach for classifying different brain consciousness states under anesthesia has been much less investigated. In this paper, we propose a new framework based on meta-learning using deep neural networks, named Anes-MetaNet, to classify brain states under anesthetics. The Anes-MetaNet is composed of Convolutional Neural Networks (CNN) to extract power spectrum features, and a time consequence model based on Long Short-Term Memory (LSTM) networks to capture the temporal dependencies, and a meta-learning framework to handle large cross-subject variability. We use a multi-stage training paradigm to improve the performance, which is justified by visualizing the high-level feature mapping. Experiments on the office-based anesthesia EEG dataset demonstrate the effectiveness of our proposed Anes-MetaNet by comparison of existing methods. [ABSTRACT FROM AUTHOR]

Most existing deep learning (DL) based direction-of-arrival (DOA) estimation methods treat direction finding problem as a multi-label classification task and the output of the neural network is a probability spectrum where the peaks indicate the true DOAs. These methods essentially belong to grid-based methods and may encounter grid mismatch effect. In this paper, we focus on gridless DL based DOA estimation under generalized linear array which can be regarded as a uniform linear array (ULA) with/without “holes”. By using the Toeplitz structure, a deep convolutional neural network (CNN) is proposed to estimate the noiseless covariance matrix of the aforementioned ULA with “no holes,” based on which the DOAs can be retrieved by using root-MUSIC. To increase the generalization, the parameters of the CNN with respect to different number of sources are pre-trained and stored in a database. We then propose another CNN for source enumeration in order to choose suitable parameters from the database. Our method can find more sources than sensors and do not suffer from the grid mismatch effect. [ABSTRACT FROM AUTHOR]

With the ongoing worldwide coronavirus disease 2019 (COVID-19) pandemic, it is desirable to develop effective algorithms to automatically detect COVID-19 with chest computed tomography (CT) images. Recently, a considerable number of methods based on deep learning have indeed been proposed. However, training an accurate deep learning model requires a large-scale chest CT dataset, which is hard to collect due to the high contagiousness of COVID-19. To achieve improved detection performance, this paper proposes a hybrid framework that fuses the complex shearlet scattering transform (CSST) and a suitable convolutional neural network into a single model. The introduced CSST cascades complex shearlet transforms with modulus nonlinearities and low-pass filter convolutions to compute a sparse and locally invariant image representation. The features computed from the input chest CT images are discriminative for COVID-19 detection. Furthermore, a wide residual network with a redesigned residual block (WR2N) is developed to learn more granular multiscale representations by applying it to scattering features. The combination of model-based CSST and data-driven WR2N leads to a more convenient neural network for image representation, where the idea is to learn only the image parts that the CSST cannot handle instead of all parts. Experiments on two public datasets demonstrate the superiority of our method. We can obtain more accurate results than several state-of-the-art COVID-19 classification methods in terms of measures such as accuracy, the F1-score, and the area under the receiver operating characteristic curve. [ABSTRACT FROM AUTHOR]

This paper proposes a generic method to learn interpretable convolutional filters in a deep convolutional neural network (CNN) for object classification, where each interpretable filter encodes features of a specific object part. Our method does not require additional annotations of object parts or textures for supervision. Instead, we use the same training data as traditional CNNs. Our method automatically assigns each interpretable filter in a high conv-layer with an object part of a certain category during the learning process. Such explicit knowledge representations in conv-layers of the CNN help people clarify the logic encoded in the CNN, i.e., answering what patterns the CNN extracts from an input image and uses for prediction. We have tested our method using different benchmark CNNs with various architectures to demonstrate the broad applicability of our method. Experiments have shown that our interpretable filters are much more semantically meaningful than traditional filters. [ABSTRACT FROM AUTHOR]

The informativeness and discriminativeness of features collaboratively ensure high-accuracy Facial Expression Recognition (FER) in the wild. Most of existing methods use the single-path deep convolutional neural network with softmax loss for basic FER, while they cannot deal with the challenging situations of the compound FER in the wild, because they fail to learn informative and discriminative features in a targeted manner. To this end, we present an Informative and Discriminative Feature Learning (IDFL) framework that consists of two key components: the Multi-Path Attention Convolutional Neural Network (MPACNN) and Balanced Separate loss (BS loss), for both basic and compound high-accuracy FER in the wild. Specifically, MPACNN leverages different paths to learn diverse features. These features are then adaptively fused into informative ones via an attention module, such that the model can adequately capture detailed information for both basic and compound FER. The BS loss maximizes the inter-class distance of features and minimizes the intra-class one. In this way, the features are discriminative enough for high-accuracy FER in the wild. Particularly, the BS loss is invoked as the objective function of MPACNN, so the model can learn informative and discriminative features at the same time, yielding better performance. Seven databases are utilized to evaluate the proposed method, and the results demonstrate that our method achieves state-of-the-art performance on both basic and compound expressions with good generalization ability. Moreover, our model contains fewer parameters and can be trained faster than other related models. [ABSTRACT FROM AUTHOR]

Wireless communications systems are impacted by multi-path fading and Doppler shift in dynamic environments, where the channel becomes doubly-dispersive and its estimation becomes an arduous task. Only a few pilots are used for channel estimation in conventional approaches to preserve high data rate transmission. Consequently, such estimators experience a significant performance degradation in high mobility scenarios. Recently, deep learning has been employed for doubly-dispersive channel estimation due to its low-complexity, robustness, and good generalization ability. Against this backdrop, the current paper presents a comprehensive survey on channel estimation techniques based on deep learning by deeply investigating different methods. The study also provides extensive experimental simulations followed by a computational complexity analysis. After considering different parameters such as modulation order, mobility, frame length, and deep learning architecture, the performance of the studied estimators is evaluated in several mobility scenarios. In addition, the source codes are made available online in order to make the results reproducible.

Deep neural network based methods are the state of the art in various image restoration problems. Standard supervised learning frameworks require a set of noisy measurement and clean image pairs for which a distance between the output of the restoration model and the ground truth, clean images is minimized. The ground truth images, however, are often unavailable or very expensive to acquire in real-world applications. We circumvent this problem by proposing a class of structured denoisers that can be decomposed as the sum of a nonlinear image-dependent mapping, a linear noise-dependent term and a small residual term. We show that these denoisers can be trained with only noisy images under the condition that the noise has zero mean and known variance. The exact distribution of the noise, however, is not assumed to be known. We show the superiority of our approach for image denoising, and demonstrate its extension to solving other restoration problems such as image deblurring where the ground truth is not available. Our method outperforms some recent unsupervised and self-supervised deep denoising models that do not require clean images for their training. For deblurring problems, the method, using only one noisy and blurry observation per image, reaches a quality not far away from its fully supervised counterparts on a benchmark dataset. [ABSTRACT FROM AUTHOR]

With the aggressive and amazing scaling of the feature size of semiconductors, hotspot detection has become a crucial and challenging problem in the generation of optimized mask design for better printability. Machine learning techniques, especially deep learning, have attained notable success on hotspot detection tasks. However, most existing hotspot detectors suffer from suboptimal performance due to two-stage flow and less efficient representations of layout features. What is more, most works can only solve simple benchmarks with apparent hotspot patterns like ICCAD 2012 Contest benchmarks. In this article, we first develop a new end-to-end hotspot detection flow where layout feature embedding and hotspot detection are jointly performed. An attention mechanism-based deep convolutional neural network (CNN) is exploited as the backbone to learn embeddings for layout features and classify the hotspots simultaneously. The experimental results demonstrate that our framework achieves accuracy improvement over prior arts with fewer false alarms and faster inference speed on much more challenging benchmarks. [ABSTRACT FROM AUTHOR]

The accurate detection and quantification of submerged targets has been recognized as a key challenge in marine exploration, one that traditional census approaches cannot handle efficiently. Here we present a deep learning approach to detect the pattern of a moving fish from the reflections of an active acoustic emitter. To allow for real-time detection, we use a convolutional neural network, which provides the simultaneous labeling of a large buffer of signal samples. This allows to capture the structure of the reflecting signal from the moving target and to separate it from clutter reflections. We evaluate system performance both on synthetic (simulated) data, as well as on real data recorded over 50 sea experiments in a variety of sea conditions. When tested on real signals, the network trained on simulated patterns showed non-trivial detection capabilities, suggesting that transfer learning can be a viable approach in these scenarios, where tagged data is often lacking. However, training the network directly on the real reflections with data augmentation techniques allowed to reach a more favorable precision-recall trade-off, approaching an ideal detection bound. We also evaluate an alternative model based on recurrent neural networks which, despite exhibiting slightly inferior performance, could be applied in scenarios requiring on-line processing of the reflection sequence. [ABSTRACT FROM AUTHOR]

This work proposes a novel deep learning (DL) framework to solve the electromagnetic inverse scattering (EMIS) problems. The proposed framework integrates the complex-valued deep convolutional neural network (DConvNet) into the supervised descent method (SDM) to realize both off-line training and on-line “imaging” prediction for EMIS. The offline training consists of two parts: 1) DConvNet training: the training dataset is created, and the proposed DConvNet is trained to realize the EM forward process and 2) SDM training: the trained DConvNet is integrated into the SDM framework, and the average descent directions between the initial prediction and the true label of SDM iterative schemes are learned based on the same dataset in part 1). In the online step, the contrasts (permittivities) reconstruction of scatterers is realized by the SDM iteration process based on learned descent directions, while its forward process is achieved by the trained complex-valued DConvNet. Ultimately, this framework provides a new perspective to integrate the prior information into the EMIS solving process with the maintained accuracy. Unlike the conventional SDM, the novel proposed framework can significantly shorten the computation and realize the real-time imaging. Various numerical examples and discussions are provided to demonstrate the efficiency and accuracy of the proposed novel framework. [ABSTRACT FROM AUTHOR]

Satellite-based synthetic aperture radar (SAR) can provide low-cost, frequent environment monitoring for dynamic intertidal zones. The critical problem is to realize pixel-level classification of SAR images of the intertidal zones with excellent and robust performance. Recently, deep learning, in particular deep convolutional neural networks, has provided us with promising solutions to this problem. Based on a sophisticated deep learning-based pixel-level classification model $\text{U}^{2}$ -Net, we propose an MB- $\text{U}^{2}$ -ACNet model suitable for intertidal zone land cover classification using dual-polarimetric SAR data integrated with environmental information, such as wind speed and tide level information. The MB- $\text{U}^{2}$ -ACNet model has a multibranch nested U-shaped encoding–decoding structure. We extract and fuse features from multiple data sources, including satellite remote sensing and environmental information, by establishing the multibranch structure. Furthermore, we propose an asymmetric convolution residual U-block for each encoding–decoding stage to improve the model’s feature extraction ability. Moreover, the model with attention mechanisms better distinguishes the importance of features from the channel’s perspectives and spatial dimensions. We construct a dataset with 106 Sentinel-1 SAR images from 2016 to 2020 for environmental monitoring in the intertidal zone of Shanghai Nanhui. On the dataset, the proposed model reaches the overall classification accuracy of 96.40% and the mean intersection over union score of 0.8307. The experiments show the advantages of the proposed model compared with the benchmarking models due to better feature extraction and multisource information fusion. In addition, the contributions of every added substructure are analyzed systematically. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs), regardless of their impressive performance, are vulnerable to attacks from adversarial inputs and, more recently, Trojans to misguide or hijack the decision of the model. We expose the existence of an intriguing class of spatially bounded, physically realizable, adversarial examples— Universal NaTuralistic adversarial paTches—we call TnTs, by exploring the super set of the spatially bounded adversarial example space and the natural input space within generative adversarial networks. Now, an adversary can arm themselves with a patch that is naturalistic, less malicious-looking, physically realizable, highly effective—achieving high attack success rates, and universal. A TnT is universal because any input image captured with a TnT in the scene will: i) misguide a network (untargeted attack); or ii) force the network to make a malicious decision (targeted attack). Interestingly, now, an adversarial patch attacker has the potential to exert a greater level of control—the ability to choose a location independent, natural-looking patch as a trigger in contrast to being constrained to noisy perturbations—an ability is thus far shown to be only possible with Trojan attack methods needing to interfere with the model building processes to embed a backdoor at the risk discovery; but, still realize a patch deployable in the physical world. Through extensive experiments on the large-scale visual classification task, ImageNet with evaluations across its entire validation set of 50,000 images, we demonstrate the realistic threat from TnTs and the robustness of the attack. We show a generalization of the attack to create patches achieving higher attack success rates than existing state-of-the-art methods. Our results show the generalizability of the attack to different visual classification tasks (CIFAR-10, GTSRB, PubFig) and multiple state-of-the-art deep neural networks such as WideResnet50, Inception-V3 and VGG-16. [ABSTRACT FROM AUTHOR]

Image matting is widely studied for accurate foreground extraction. Most algorithms, including deep-learning based solutions, require a carefully edited trimap. Recent works attempt to combine the segmentation stage and matting stage in one CNN model, but errors occurring at the segmentation stage lead to unsatisfactory matte. We propose a user-guided approach for practical human matting. More precisely, we provide a good automatic initial matting and a natural way of interaction that reduces the workload of drawing trimaps and allows users to guide the matting in ambiguous situation. We also combine the segmentation and matting stage in an end-to-end CNN architecture and introduce a residual-learning module to support convenient stroke-based interaction. The proposed model learns to propagate the input trimap and modify the deep image features, which can efficiently correct the segmentation errors. Our model supports arbitrary forms of trimaps from carefully edited to totally unknown maps. Our model also allows users to choose from different foreground estimations according to their preference. We collected a large human matting dataset consisting of 12K real-world human images with complex background and human-object relations. The proposed model is trained on the new dataset with a novel trimap generation strategy that enables the model to tackle different test situations and highly improves the interaction efficiency. Our method outperforms other state-of-the-art automatic methods and achieve competitive accuracy when high-quality trimaps are provided. Experiments indicate that our interactive matting strategy is superior to separately estimating the trimap and alpha matte using two models. [ABSTRACT FROM AUTHOR]

Unavailability of large training datasets is a bottleneck that needs to be overcome to realize the true potential of deep learning in histopathology applications. Although slide digitization via whole slide imaging scanners has increased the speed of data acquisition, labeling of virtual slides requires a substantial time investment from pathologists. Eye gaze annotations have the potential to speed up the slide labeling process. This work explores the viability and timing comparisons of eye gaze labeling compared to conventional manual labeling for training object detectors. Challenges associated with gaze based labeling and methods to refine the coarse data annotations for subsequent object detection are also discussed. Results demonstrate that gaze tracking based labeling can save valuable pathologist time and delivers good performance when employed for training a deep object detector. Using the task of localization of Keratin Pearls in cases of oral squamous cell carcinoma as a test case, we compare the performance gap between deep object detectors trained using hand-labelled and gaze-labelled data. On average, compared to ‘Bounding-box’ based hand-labeling, gaze-labeling required 57.6% less time per label and compared to ‘Freehand’ labeling, gaze-labeling required on average 85% less time per label. [ABSTRACT FROM AUTHOR]

This research proposes a mobile and cloud-based framework for the automatic detection of diabetic foot ulcers and conducts an investigation of its performance. The system uses a cross-platform mobile framework that enables the deployment of mobile apps to multiple platforms using a single TypeScript code base. A deep convolutional neural network was deployed to a cloud-based platform where the mobile app could send photographs of patient's feet for inference to detect the presence of diabetic foot ulcers. The functionality and usability of the system were tested in two clinical settings: Salford Royal NHS Foundation Trust and Lancashire Teaching Hospitals NHS Foundation Trust. The benefits of the system, such as the potential use of the app by patients to identify and monitor their condition, are discussed. [ABSTRACT FROM AUTHOR]

The lack of labels is one of the major challenges in hyperspectral image (HSI) classification. Widely used Deep Learning (DL) models such as convolutional neural networks (CNNs) experience serious performance degradation when training samples are limited. In contrast, graph convolutional networks (GCNs) can simultaneously exploit the insufficient labeled data and massive unlabeled data of HSI in a semisupervised learning fashion. However, in order to reduce computational cost and mitigate noise, existing GCN-based classification methods usually perform superpixel segmentation as a preprocessing step and implement feature extraction as well as node classification on the predefined superpixel graph, where one superpixel might incorporate pixels with different labels. Moreover, the local spectral–spatial information within superpixels is generally ignored. To alleviate these two issues, we propose an end-to-end mixhop superpixel-based GCN (EMS-GCN) framework for HSI classification. Specifically, we first introduce the differentiable superpixel segmentation algorithm to map the pixel representations into a superpixel feature space, which allows refining the superpixel boundary with the training of the network. After that, a superpixel graph is constructed and fed into a novel mixhop superpixel-based GCN, where both the local information within superpixels and long-range information among superpixels are extracted, while the structure of the superpixel graph is updated at the same time. Finally, the enhanced superpixel representations are mapped back into a pixel feature space to conduct pixel-wise classification. Extensive experiments demonstrate the effectiveness of the proposed EMS-GCN method compared with other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Fused magnesium furnace (FMF) is the vital equipment for magnesia refractory production. The melting process of FMF is subject to melting temperature, magnesite quality and composition, and much more dynamic factors, which are prone to abnormal conditions such as overheating, abnormal exhausting, semi-molten, and affect the stability and safety of the production seriously. Due to the difficulties of effectively monitoring and accurately identifying the abnormal conditions with the existing methods, this article proposes a novel method for identifying the abnormal conditions of FMF based on deep learning and multi-information fusion. First, the Wasserstein distance based generative adversarial networks with gradient penalty is developed to solve the problem of obtaining overheating condition image samples with obvious visual characteristics. Then, the image-based deep convolutional neural network model is established to recognize the abnormal condition images and extract image features whose training set is the combination of the generated images and the original images. Finally, based on multisource information fusion, the traditional process current data are fused with the extracted image features, and a support vector machine model is established to complete the classification and identification of the FMF working conditions. Various experiments on actual industrial data show that the proposed method is effective and advanced. [ABSTRACT FROM AUTHOR]

Lithology interpretation is important for understanding subsurface properties. Yet, the common manual well log interpretation is usually with low efficiency and bad consistency. Therefore, the automatic well log interpretation tools based on machine learning and deep learning have been developed. Although the state-of-the-art sophisticated models can show fine interpretation performance with acceptable accuracies, “blind” tests do not always exhibit satisfactory results because of the complexity of lithology interpretation with respect to subsurface rock properties and the data-labeling quality. To solve this generalization challenge, we propose to leverage the parameterized quantum circuits in the deep-learning model. The quantum computing takes advantages of the superposition and entanglement quantum systems, which could potentially endow the generalization power or capability to the deep-learning model. Using the proposed quantum-enhanced deep-learning (QEDL) model, we have tested the model performance on field well log data from different wells. Compared with the classic fine convolutional neural network (CNN) model and the long short-term memory (LSTM) model, the proposed QEDL model achieves comparable model performance with a clearly improved generalization power for interpreting both thin and thick lithology layers. In addition, because of the quantum circuit structure, the QEDL model needs much fewer model parameters than LSTM and CNN models, i.e., the QEDL parameter number in our study can be approximately 75% less than that of LSTM and 89% less than that of CNN. [ABSTRACT FROM AUTHOR]

Large-scale distributed training of deep neural networks results in models with worse generalization performance as a result of the increase in the effective mini-batch size. Previous approaches attempt to address this problem by varying the learning rate and batch size over epochs and layers, or ad hoc modifications of batch normalization. We propose scalable and practical natural gradient descent (SP-NGD), a principled approach for training models that allows them to attain similar generalization performance to models trained with first-order optimization methods, but with accelerated convergence. Furthermore, SP-NGD scales to large mini-batch sizes with a negligible computational overhead as compared to first-order methods. We evaluated SP-NGD on a benchmark task where highly optimized first-order methods are available as references: training a ResNet-50 model for image classification on ImageNet. We demonstrate convergence to a top-1 validation accuracy of 75.4 percent in 5.5 minutes using a mini-batch size of 32,768 with 1,024 GPUs, as well as an accuracy of 74.9 percent with an extremely large mini-batch size of 131,072 in 873 steps of SP-NGD. [ABSTRACT FROM AUTHOR]

Deep learning is heavily being borrowed to solve problems in medical imaging applications, and Siamese neural networks are the front runners of motion tracking. In this article, we propose to upgrade one such Siamese architecture-based neural network for robust and accurate landmark tracking in ultrasound images to improve the quality of image-guided radiation therapy. Although several researchers have improved the Siamese architecture-based networks with sophisticated detection modules and by incorporating transfer learning, the inherent assumptions of the constant position model and the missing motion model remain unaddressed limitations. In our proposed model, we overcome these limitations by introducing two modules into the original architecture. We employ a reference template update to resolve the constant position model and a linear Kalman filter (LKF) to address the missing motion model. Moreover, we demonstrate that the proposed architecture provides promising results without transfer learning. The proposed model was submitted to an open challenge organized by MICCAI and was evaluated exhaustively on the Liver US Tracking (CLUST) 2D dataset. Experimental results proved that the proposed model tracked the landmarks with promising accuracy. Furthermore, we also induced synthetic occlusions to perform a qualitative analysis of the proposed approach. The evaluations were performed on the training set of the CLUST 2D dataset. The proposed method outperformed the original Siamese architecture by a significant margin. [ABSTRACT FROM AUTHOR]