Search

Stereo matching methods consist of matching cost computation and several post processing steps. Deep learning methods have greatly raised the accuracy of matching cost and achieved the lowest error rate on several public datasets. However, their generality capabilities are not the best due to potential overfitting, which is the common problem of supervised learning approaches. This paper proposes a convolutional neural network (CNN) based cost estimation method for computing the similarity of image patches. In consideration of accuracy and generalization capability, small size convolution kernels are chosen in the convolution layer and dropout in the decision layer is used for preventing overfitting. After obtaining stereo matching cost from the output of the CNN, several post-processing operations are adopted for disparity optimization, which includes semi-global matching in 1-D from different directions, a left-right consistency check, and the slanted plane smoothing method. The method is evaluated on KITTI 2012, KITTI 2015, and Middlebury stereo datasets and the experimental results on the KITTI benchmark demonstrate the competitive accuracy performance of the approach. Additionally, to test the generalization of the method, a series of extended crossover experiments are conducted in which the training samples and testing samples come from different datasets. The results indicate superior generalization capability of our method than other supervised learning methods. [ABSTRACT FROM AUTHOR]

This paper proposes a novel method for voltage dip classification using deep convolutional neural networks. The main contributions of this paper include: 1) to propose a new effective deep convolutional neural network architecture for automatically learning voltage dip features, rather than extracting hand-crafted features; 2) to employ the deep learning in an effective two-dimensional (2-D) transform domain, under space-phasor model (SPM), for efficient learning of dip features; 3) to characterize voltage dips by 2-D SPM-based deep learning, which leads to voltage dip features independent of the duration and sampling frequency of dip recordings; and 4) to develop robust automatically-extracted features that are insensitive to training and test datasets measured from different countries/regions. Experiments were conducted on datasets containing about 6000 measured voltage dips spread over seven classes measured from several different countries. Results have shown good performance of the proposed method: average classification rate is about 97% and false alarm rate is about 0.50%. The test results from the proposed method are compared with the results from two existing dip classification methods. The proposed method is shown to outperform these existing methods. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

X-ray computed tomography (CT) using sparse projection views is a recent approach to reduce the radiation dose. However, due to the insufficient projection views, an analytic reconstruction approach using the filtered back projection (FBP) produces severe streaking artifacts. Recently, deep learning approaches using large receptive field neural networks such as U-Net have demonstrated impressive performance for sparse-view CT reconstruction. However, theoretical justification is still lacking. Inspired by the recent theory of deep convolutional framelets, the main goal of this paper is, therefore, to reveal the limitation of U-Net and propose new multi-resolution deep learning schemes. In particular, we show that the alternative U-Net variants such as dual frame and tight frame U-Nets satisfy the so-called frame condition which makes them better for effective recovery of high frequency edges in sparse-view CT. Using extensive experiments with real patient data set, we demonstrate that the new network architectures provide better reconstruction performance. [ABSTRACT FROM AUTHOR]

Graph Neural Networks (GNNs) have led to state-of-the-art performance on a variety of machine learning tasks such as recommendation, node classification and link prediction. Graph neural network models generate node embeddings by merging nodes features with the aggregated neighboring nodes information. Most existing GNN models exploit a single type of aggregator (e.g., mean-pooling) to aggregate neighboring nodes information, and then add or concatenate the output of aggregator to the current representation vector of the center node. However, using only a single type of aggregator is difficult to capture the different aspects of neighboring information and the simple addition or concatenation update methods limit the expressive capability of GNNs. Not only that, existing supervised or semi-supervised GNN models are trained based on the loss function of the node label, which leads to the neglect of graph structure information. In this paper, we propose a novel graph neural network architecture, Graph Attention & Interaction Network (GAIN), for inductive learning on graphs. Unlike the previous GNN models that only utilize a single type of aggregation method, we use multiple types of aggregators to gather neighboring information in different aspects and integrate the outputs of these aggregators through the aggregator-level attention mechanism. Furthermore, we design a graph regularized loss to better capture the topological relationship of the nodes in the graph. Additionally, we first present the concept of graph feature interaction and propose a vector-wise explicit feature interaction mechanism to update the node embeddings. We conduct comprehensive experiments on two node-classification benchmarks and a real-world financial news dataset. The experiments demonstrate our GAIN model outperforms current state-of-the-art performances on all the tasks. [ABSTRACT FROM AUTHOR]

The purpose of this paper is the development of data science models for the detection of empty on-street parking spaces in urban road networks based on data provided by in-vehicle cameras that are already, or soon will be, a standard vehicle equipment. A rolling spatial interval is used to identify the existence of an on-street parking space and the properties of empty spaces are used to determine the availability of the parking space. Convolutional neural networks are developed, trained, and evaluated with the use of images from a moving vehicle camera. The images are preprocessed and converted to suitable matrices, so that only the useful information for the empty on-street parking space detection problem is preserved. The optimized convolutional networks, in terms of structural and learning parameters, provided predictions for the detection of empty on-street parking spaces with approximately 90% average accuracy. The proposed model performs better than the relatively complex SVMs, which supports its appropriateness as an approach. Finally, the implementation of a framework, which integrates the developed models to produce meaningful parking information for drivers in real time, is discussed. [ABSTRACT FROM AUTHOR]

The segmentation of multiple sclerosis (MS) lesions from MR imaging sequences remains a challenging task, due to the characteristics of variant shapes, scattered distributions and unknown numbers of lesions. However, the current automated MS segmentation methods with deep learning models face the challenges of (1) capturing the scattered lesions in multiple regions and (2) delineating the global contour of variant lesions. To address these challenges, in this paper, we propose a novel attention and graph-driven network (DAG-Net), which incorporates (1) the spatial correlations for embracing the lesions in distant regions and (2) the global context for better representing lesions of variant features in a unified architecture. Firstly, the novel local attention coherence mechanism is designed to construct dynamic and expansible graphs for the spatial correlations between pixels and their proximities. Secondly, the proposed spatial-channel attention module enhances features to optimize the global contour delineation, by aggregating relevant features. Moreover, with the dynamic graphs, the learning process of the DAG-Net is interpretable, which in turns support the reliability of segmentation results. Extensive experiments were conducted on a public ISBI2015 dataset and an in-house dataset in comparison to state-of-the-art methods, based on geometrical and clinical metrics. The experimental results validate the effectiveness of proposed DAG-Net on segmenting variant and scatted lesions in multiple regions. [ABSTRACT FROM AUTHOR]

Designing volume visualizations showing various structures of interest is critical to the exploratory analysis of volumetric data. The last few years have witnessed dramatic advances in the use of convolutional neural networks for identification of objects in large image collections. Whereas such machine learning methods have shown superior performance in a number of applications, their direct use in volume visualization has not yet been explored. In this paper, we present a deep-learning-assisted volume visualization to depict complex structures, which are otherwise challenging for conventional approaches. A significant challenge in designing volume visualizations based on the high-dimensional deep features lies in efficiently handling the immense amount of information that deep-learning methods provide. In this paper, we present a new technique that uses spectral methods to facilitate user interactions with high-dimensional features. We also present a new deep-learning-assisted technique for hierarchically exploring a volumetric dataset. We have validated our approach on two electron microscopy volumes and one magnetic resonance imaging dataset. [ABSTRACT FROM AUTHOR]

This paper presents an energy-efficient static random access memory (SRAM) with embedded dot-product computation capability, for binary-weight convolutional neural networks. A 10T bit-cell-based SRAM array is used to store the 1-b filter weights. The array implements dot-product as a weighted average of the bitline voltages, which are proportional to the digital input values. Local integrating analog-to-digital converters compute the digital convolution outputs, corresponding to each filter. We have successfully demonstrated functionality (>98% accuracy) with the 10 000 test images in the MNIST hand-written digit recognition data set, using 6-b inputs/outputs. Compared to conventional full-digital implementations using small bitwidths, we achieve similar or better energy efficiency, by reducing data transfer, due to the highly parallel in-memory analog computations. [ABSTRACT FROM AUTHOR]

Person re-identification (Re-ID) aims to match person images captured from two non-overlapping cameras. In this paper, a deep hybrid similarity learning (DHSL) method for person Re-ID based on a convolution neural network (CNN) is proposed. In our approach, a light CNN learning feature pair for the input image pair is simultaneously extracted. Then, both the elementwise absolute difference and multiplication of the CNN learning feature pair are calculated. Finally, a hybrid similarity function is designed to measure the similarity between the feature pair, which is realized by learning a group of weight coefficients to project the elementwise absolute difference and multiplication into a similarity score. Consequently, the proposed DHSL method is able to reasonably assign complexities of feature learning and metric learning in a CNN, so that the performance of person Re-ID is improved. Experiments on three challenging person Re-ID databases, QMUL GRID, VIPeR, and CUHK03, illustrate that the proposed DHSL method is superior to multiple state-of-the-art person Re-ID methods. [ABSTRACT FROM AUTHOR]

Automatic modulation classification (AMC), which plays critical roles in both civilian and military applications, is investigated in this paper through a deep learning approach. Conventional AMCs can be categorized into maximum likelihood (ML) based (ML-AMC) and feature-based AMC. However, the practical deployment of ML-AMCs is difficult due to its high computational complexity, and the manually extracted features require expert knowledge. Therefore, an end-to-end convolution neural network (CNN) based AMC (CNN-AMC) is proposed, which automatically extracts features from the long symbol-rate observation sequence along with the estimated signal-to-noise ratio (SNR). With CNN-AMC, a unit classifier is adopted to accommodate the varying input dimensions. The direct training of CNN-AMC is challenging with the complicated model and complex tasks, so a novel two-step training is proposed, and the transfer learning is also introduced to improve the efficiency of retraining. Different digital modulation schemes have been considered in distinct scenarios, and the simulation results show that the CNN-AMC can outperform the feature-based method, and obtain a closer approximation to the optimal ML-AMC. Besides, CNN-AMCs have the certain robustness to estimation error on carrier phase offset and SNR. With parallel computation, the deep-learning-based approach is about $ 40$ to $ 1700$ times faster than the ML-AMC regarding inference speed. [ABSTRACT FROM AUTHOR]

Semantic segmentation requires both speed and accuracy. This paper presents a two-branch network BiAttnNet with a unique Bilateral Attention structure that separates all attention modules into the Detail Branch to contribute semantic detail selections for specialized detail exploring. Specifically, the Detail Branch comprises AttnTrans entirely, which provides a better alternate for regular convolution. AttnTrans is a computationally efficient filtration entirely composed of concurrent spatial and channel attention. Meanwhile, a Context Branch is implemented with FCN-ResNet for rough segmentation. By combining two branches’ outputs, BiAttnNet achieves a good balance between speed and accuracy. Evaluations on the Cityscapes testing set conclude that BiAttnNet achieves 74.7% mIoU at 89.2 FPS at a quarter (512 × 1024) resolution with only 2.2 million parameters, running on a single GTX 2080 Ti card. [ABSTRACT FROM AUTHOR]

Emotion recognition is challenging due to the emotional gap between emotions and audio–visual features. Motivated by the powerful feature learning ability of deep neural networks, this paper proposes to bridge the emotional gap by using a hybrid deep model, which first produces audio–visual segment features with Convolutional Neural Networks (CNNs) and 3D-CNN, then fuses audio–visual segment features in a Deep Belief Networks (DBNs). The proposed method is trained in two stages. First, CNN and 3D-CNN models pre-trained on corresponding large-scale image and video classification tasks are fine-tuned on emotion recognition tasks to learn audio and visual segment features, respectively. Second, the outputs of CNN and 3D-CNN models are combined into a fusion network built with a DBN model. The fusion network is trained to jointly learn a discriminative audio–visual segment feature representation. After average-pooling segment features learned by DBN to form a fixed-length global video feature, a linear Support Vector Machine is used for video emotion classification. Experimental results on three public audio–visual emotional databases, including the acted RML database, the acted eNTERFACE05 database, and the spontaneous BAUM-1s database, demonstrate the promising performance of the proposed method. To the best of our knowledge, this is an early work fusing audio and visual cues with CNN, 3D-CNN, and DBN for audio–visual emotion recognition. [ABSTRACT FROM AUTHOR]

Deep learning has achieved great successes in conventional computer vision tasks. In this paper, we exploit deep learning techniques to address the hyperspectral image classification problem. In contrast to conventional computer vision tasks that only examine the spatial context, our proposed method can exploit both spatial context and spectral correlation to enhance hyperspectral image classification. In particular, we advocate four new deep learning models, namely, 2-D convolutional neural network (2-D-CNN), 3-D-CNN, recurrent 2-D CNN (R-2-D-CNN), and recurrent 3-D-CNN (R-3-D-CNN) for hyperspectral image classification. We conducted rigorous experiments based on six publicly available data sets. Through a comparative evaluation with other state-of-the-art methods, our experimental results confirm the superiority of the proposed deep learning models, especially the R-3-D-CNN and the R-2-D-CNN deep learning models. [ABSTRACT FROM AUTHOR]

Many practical machine learning tasks employ very deep convolutional neural networks. Such large depths pose formidable computational challenges in training and operating the network. It is therefore important to understand how fast the energy contained in the propagated signals (a.k.a. feature maps) decays across layers. In addition, it is desirable that the feature extractor generated by the network be informative in the sense of the only signal mapping to the all-zeros feature vector being the zero input signal. This “trivial null-set” property can be accomplished by asking for “energy conservation” in the sense of the energy in the feature vector being proportional to that of the corresponding input signal. This paper establishes conditions for energy conservation (and thus for a trivial null-set) for a wide class of deep convolutional neural network-based feature extractors and characterizes corresponding feature map energy decay rates. Specifically, we consider general scattering networks employing the modulus non-linearity and we find that under mild analyticity and high-pass conditions on the filters (which encompass, inter alia, various constructions of Weyl-Heisenberg filters, wavelets, ridgelets, ( $\alpha $ )-curvelets, and shearlets) the feature map energy decays at least polynomially fast. For broad families of wavelets and Weyl-Heisenberg filters, the guaranteed decay rate is shown to be exponential. Moreover, we provide handy estimates of the number of layers needed to have at least $((1-\varepsilon )\cdot 100)\%$ of the input signal energy be contained in the feature vector. [ABSTRACT FROM AUTHOR]

The key challenge of intelligent fault diagnosis is to develop features that can distinguish different categories. Because of the unique properties of mechanical data, predetermined features based on prior knowledge are usually used as inputs for fault classification. However, proper selection of features often requires expertise knowledge and becomes more difficult and time consuming when volume of data increases. In this paper, a novel deep learning network (LiftingNet) is proposed to learn features adaptively from raw mechanical data without prior knowledge. Inspired by convolutional neural network and second generation wavelet transform, the LiftingNet is constructed to classify mechanical data even though inputs contain considerable noise and randomness. The LiftingNet consists of split layer, predict layer, update layer, pooling layer, and full-connection layer. Different kernel sizes are allowed in convolutional layers to improve learning ability. As a multilayer neural network, deep features are learned from shallow ones to represent complex structures in raw data. Feasibility and effectiveness of the LiftingNet is validated by two motor bearing datasets. Results show that the proposed method could achieve layerwise feature learning and successfully classify mechanical data even with different rotating speed and under the influence of random noise. [ABSTRACT FROM PUBLISHER]

Deep convolutional neural networks (DCNNs) have led to breakthrough results in numerous practical machine learning tasks, such as classification of images in the ImageNet data set, control-policy-learning to play Atari games or the board game Go, and image captioning. Many of these applications first perform feature extraction and then feed the results thereof into a classifier. The mathematical analysis of DCNNs for feature extraction was initiated by Mallat, 2012. Specifically, Mallat considered so-called scattering networks based on a wavelet transform followed by the modulus non-linearity in each network layer, and proved translation invariance (asymptotically in the wavelet scale parameter) and deformation stability of the corresponding feature extractor. This paper complements Mallat’s results by developing a theory that encompasses general convolutional transforms, or in more technical parlance, general semi-discrete frames (including Weyl–Heisenberg filters, curvelets, shearlets, ridgelets, wavelets, and learned filters), general Lipschitz-continuous non-linearities (e.g., rectified linear units, shifted logistic sigmoids, hyperbolic tangents, and modulus functions), and general Lipschitz-continuous pooling operators emulating, e.g., sub-sampling and averaging. In addition, all of these elements can be different in different network layers. For the resulting feature extractor, we prove a translation invariance result of vertical nature in the sense of the features becoming progressively more translation-invariant with increasing network depth, and we establish deformation sensitivity bounds that apply to signal classes such as, e.g., band-limited functions, cartoon functions, and Lipschitz functions. [ABSTRACT FROM PUBLISHER]

Detection algorithms for electroencephalography (EEG) data, especially in the field of interictal epileptiform discharge (IED) detection, have traditionally employed handcrafted features, which utilized specific characteristics of neural responses. Although these algorithms achieve high accuracy, mere detection of an IED holds little clinical significance. In this paper, we consider deep learning for epileptic subjects to accommodate automatic feature generation from intracranial EEG data, while also providing clinical insight. Convolutional neural networks are trained in a subject independent fashion to demonstrate how meaningful features are automatically learned in a hierarchical process. We illustrate how the convolved filters in the deepest layers provide insight toward the different types of IEDs within the group, as confirmed by our expert clinicians. The morphology of the IEDs found in filters can help evaluate the treatment of a patient. To improve the learning of the deep model, moderately different score classes are utilized as opposed to binary IED and non-IED labels. The resulting model achieves state-of-the-art classification performance and is also invariant to time differences between the IEDs. This paper suggests that deep learning is suitable for automatic feature generation from intracranial EEG data, while also providing insight into the data. [ABSTRACT FROM PUBLISHER]

The paper presents a method using deep learning framework based on convolution neural network (CNN), for identification and localization of faults of transformer winding under impulse test. The results show that the proposed method outperforms the existing methods significantly. The present scheme eliminates the requirement of separate feature extraction and classification algorithms for the analysis of fault current patterns. A part of the proposed network performs feature learning and the other part classifies the features in a supervised manner. The method is computation intensive but capable of achieving very high degree of accuracy; on an average a margin of more than 7% compared to other published literature till date. [ABSTRACT FROM AUTHOR]

Vascular tree disentanglement and vessel type classification are two crucial steps of the graph-based method for retinal artery-vein (A/V) separation. Existing approaches treat them as two independent tasks and mostly rely on ad hoc rules (e.g. change of vessel directions) and hand-crafted features (e.g. color, thickness) to handle them respectively. However, we argue that the two tasks are highly correlated and should be handled jointly since knowing the A/V type can unravel those highly entangled vascular trees, which in turn helps to infer the types of connected vessels that are hard to classify based on only appearance. Therefore, designing features and models isolatedly for the two tasks often leads to a suboptimal solution of A/V separation. In view of this, this paper proposes a multi-task siamese network which aims to learn the two tasks jointly and thus yields more robust deep features for accurate A/V separation. Specifically, we first introduce Convolution Along Vessel (CAV) to extract the visual features by convolving a fundus image along vessel segments, and the geometric features by tracking the directions of blood flow in vessels. The siamese network is then trained to learn multiple tasks: i) classifying A/V types of vessel segments using visual features only, and ii) estimating the similarity of every two connected segments by comparing their visual and geometric features in order to disentangle the vasculature into individual vessel trees. Finally, the results of two tasks mutually correct each other to accomplish final A/V separation. Experimental results demonstrate that our method can achieve accuracy values of 94.7%, 96.9%, and 94.5% on three major databases (DRIVE, INSPIRE, WIDE) respectively, which outperforms recent state-of-the-arts. [ABSTRACT FROM AUTHOR]

Learning-based face descriptors have constantly improved the face recognition performance. Compared with the hand-crafted features, learning-based features are considered to be able to exploit information with better discriminative ability for specific tasks. Motivated by the recent success of deep learning, in this paper, we extend the original shallow face descriptors to deep discriminant face features by introducing a stacked image descriptor (SID). With deep structure, more complex facial information can be extracted and the discriminant and compactness of feature representation can be improved. The SID is learned in a forward optimization way, which is computational efficient compared with deep learning. Extensive experiments on various face databases are conducted to show that SID is able to achieve high face recognition performance with compact face representation, compared with other state-of-the-art descriptors. [ABSTRACT FROM AUTHOR]

Motor imagery electroencephalography (EEG) decoding is an essential part of brain-computer interfaces (BCIs) which help motor-disabled patients to communicate with the outside world by external devices. Recently, deep learning algorithms using decomposed spectrums of EEG as inputs may omit important spatial dependencies and different temporal scale information, thus generated the poor decoding performance. In this paper, we propose an end-to-end EEG decoding framework, which employs raw multi-channel EEG as inputs, to boost decoding accuracy by the channel-projection mixed-scale convolutional neural network (CP-MixedNet) aided by amplitude-perturbation data augmentation. Specifically, the first block in CP-MixedNet is designed to learn primary spatial and temporal representations from EEG signals. The mixed-scale convolutional block is then used to capture mixed-scale temporal information, which effectively reduces the number of training parameters when expanding reception fields of the network. Finally, based on the features extracted in previous blocks, the classification block is constructed to classify EEG tasks. The experiments are implemented on two public EEG datasets (BCI competition IV 2a and High gamma dataset) to validate the effectiveness of the proposed approach compared to the state-of-the-art methods. The competitive results demonstrate that our proposed method is a promising solution to improve the decoding performance of motor imagery BCIs. [ABSTRACT FROM AUTHOR]

Deep learning workloads, such as convolutional neural networks (CNNs) are important due to increasingly demanding high-performance hardware acceleration. One distinguishing feature of a deep learning workload is that it is inherently resilient to small numerical errors and thus works very well with low precision hardware. We propose a novel method called double multiply-and-accumulate (MAC) to theoretically double the computation rate of CNN accelerators by packing two MAC operations into one digital signal processing block of off-the-shelf field-programmable gate arrays (FPGAs). We overcame several technical challenges by exploiting the mode of operation in the CNN accelerator. We have validated our method through FPGA synthesis and Verilog simulation, and evaluated our method by applying it to the state-of-the-art CNN accelerator. The double MAC approach used can double the computation throughput of a CNN layer. On the network level (all convolution layers combined), the performance improvement varies depending on the CNN application and FPGA size, from 14% to more than 80% over a highly optimized state-of-the-art accelerator solution, without sacrificing the output quality significantly. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (CNNs) have revolutionized the computer vision research and have seen unprecedented adoption for multiple tasks, such as classification, detection, and caption generation. However, they offer little transparency into their inner workings and are often treated as black boxes that deliver excellent performance. In this paper, we aim at alleviating this opaqueness of CNNs by providing visual explanations for the network’s predictions. Our approach can analyze a variety of CNN-based models trained for computer vision applications, such as object recognition and caption generation. Unlike the existing methods, we achieve this via unraveling the forward pass operation. The proposed method exploits feature dependencies across the layer hierarchy and uncovers the discriminative image locations that guide the network’s predictions. We name these locations CNN fixations, loosely analogous to human eye fixations. Our approach is a generic method that requires no architectural changes, additional training, or gradient computation, and computes the important image locations (CNN fixations). We demonstrate through a variety of applications that our approach is able to localize the discriminative image locations across different network architectures, diverse vision tasks, and data modalities. [ABSTRACT FROM AUTHOR]

This paper presents a weakly supervised regression model for the visual house appraisal problem, which aims to predict the value of a house from its photos and textual descriptions (e.g., number of bedrooms). The key idea of our approach is a multi-layer neural network, called multi-instance Deep Ranking and Regression (MiDRR) net, which jointly solves two coupled tasks: ranking and regression, in the multiple instance setting. The network is trained using weakly supervised data, which do not require intensive human annotations. We also design a set of human heuristics to promote deep features through imposing constraints over the solution space, e.g., a house with three bedrooms often has a higher value than that with only two bedrooms. While these constraints are specific to the studied problem, the developed formula can be easily generalized to the other regression applications. For test and evaluation purposes, we collect a comprehensive house image benchmark that includes 900,000 photos from 30,000 houses recently traded in the USA, and apply the proposed MiDRR net to predict house values. Extensive evaluations with comparisons demonstrate that additional usage of imagery data as well as human heuristics can significantly boost system performance and that the proposed MiDRR net clearly outperforms the alternative methods. [ABSTRACT FROM AUTHOR]

The recently proposed multilayer convolutional sparse coding (ML-CSC) model, consisting of a cascade of convolutional sparse layers, provides a new interpretation of convolutional neural networks (CNNs). Under this framework, the forward pass in a CNN is equivalent to a pursuit algorithm aiming to estimate the nested sparse representation vectors from a given input signal. Despite having served as a pivotal connection between CNNs and sparse modeling, a deeper understanding of the ML-CSC is still lacking. In this paper, we propose a sound pursuit algorithm for the ML-CSC model by adopting a projection approach. We provide new and improved bounds on the stability of the solution of such pursuit and we analyze different practical alternatives to implement this in practice. We show that the training of the filters is essential to allow for nontrivial signals in the model, and we derive an online algorithm to learn the dictionaries from real data, effectively resulting in cascaded sparse convolutional layers. Last, but not least, we demonstrate the applicability of the ML-CSC model for several applications in an unsupervised setting, providing competitive results. Our work represents a bridge between matrix factorization, sparse dictionary learning, and sparse autoencoders, and we analyze these connections in detail. [ABSTRACT FROM AUTHOR]

Model-based iterative reconstruction algorithms for low-dose X-ray computed tomography (CT) are computationally expensive. To address this problem, we recently proposed a deep convolutional neural network (CNN) for low-dose X-ray CT and won the second place in 2016 AAPM Low-Dose CT Grand Challenge. However, some of the textures were not fully recovered. To address this problem, here we propose a novel framelet-based denoising algorithm using wavelet residual network which synergistically combines the expressive power of deep learning and the performance guarantee from the framelet-based denoising algorithms. The new algorithms were inspired by the recent interpretation of the deep CNN as a cascaded convolution framelet signal representation. Extensive experimental results confirm that the proposed networks have significantly improved performance and preserve the detail texture of the original images. [ABSTRACT FROM AUTHOR]

Convolutional Neural Network (CNN) has gained attractions in image analytics and speech recognition in recent years. However, employing CNN for classification of graphs remains to be challenging. This paper presents the Ngram graph-block based convolutional neural network model for classification of graphs. Our Ngram deep learning framework consists of three novel components. First, we introduce the concept of $n$ -gram block to transform each raw graph object into a sequence of $n$ -gram blocks connected through overlapping regions. Second, we introduce a diagonal convolution step to extract local patterns and connectivity features hidden in these $n$ -gram blocks by performing $n$ -gram normalization. Finally, we develop deeper global patterns based on the local patterns and the ways that they respond to overlapping regions by building a $n$ -gram deep learning model using convolutional neural network. We evaluate the effectiveness of our approach by comparing it with the existing state of art methods using five real graph repositories from bioinformatics and social networks domains. Our results show that the Ngram approach outperforms existing methods with high accuracy and comparable performance. [ABSTRACT FROM PUBLISHER]

Many studies on the prediction of manufacturing results using sensor signals have been conducted in the field of fault detection and classification (FDC) for semiconductor manufacturing processes. However, fault diagnosis used to find clues as to root causes remains a challenging area. In particular, process monitoring using neural networks has been employed to only a limited extent because it is a black box model, making the relationships between input data and output results difficult to interpret in actual manufacturing settings, despite its high classification performance. In this paper, we propose a convolutional neural network (CNN) model, named FDC-CNN, in which a receptive field tailored to multivariate sensor signals slides along the time axis, to extract fault features. This approach enables the association of the output of the first convolutional layer with the structural meaning of the raw data, making it possible to locate the variable and time information that represents process faults. In an experiment on a chemical vapor deposition process, the proposed method outperformed other deep learning models. [ABSTRACT FROM PUBLISHER]

The annihilating filter-based low-rank Hankel matrix approach (ALOHA) is one of the state-of-the-art compressed sensing approaches that directly interpolates the missing k-space data using low-rank Hankel matrix completion. The success of ALOHA is due to the concise signal representation in the k-space domain, thanks to the duality between structured low-rankness in the k-space domain and the image domain sparsity. Inspired by the recent mathematical discovery that links convolutional neural networks to Hankel matrix decomposition using data-driven framelet basis, here we propose a fully data-driven deep learning algorithm for k-space interpolation. Our network can be also easily applied to non-Cartesian k-space trajectories by simply adding an additional regridding layer. Extensive numerical experiments show that the proposed deep learning method consistently outperforms the existing image-domain deep learning approaches. [ABSTRACT FROM AUTHOR]

The estimation of nested functions (i.e., functions of functions) is one of the central reasons for the success and popularity of machine learning. Today, artificial neural networks are the predominant class of algorithms in this area, known as representational learning. Here, we introduce Representational Gradient Boosting (RGB), a nonparametric algorithm that estimates functions with multi-layer architectures obtained using backpropagation in the space of functions. RGB does not need to assume a functional form in the nodes or output (e.g., linear models or rectified linear units), but rather estimates these transformations. RGB can be seen as an optimized stacking procedure where a meta algorithm learns how to combine different classes of functions (e.g., Neural Networks (NN) and Gradient Boosting (GB)), while building and optimizing them jointly in an attempt to compensate each other’s weaknesses. This highlights a stark difference with current approaches to meta-learning that combine models only after they have been built independently. We showed that providing optimized stacking is one of the main advantages of RGB over current approaches. Additionally, due to the nested nature of RGB we also showed how it improves over GB in problems that have several high-order interactions. Finally, we investigate both theoretically and in practice the problem of recovering nested functions and the value of prior knowledge. [ABSTRACT FROM AUTHOR]

Complex fuzzy sets (CFSs) are an extension of type-1 fuzzy sets in which the membership of an object to the set is a value from the unit disc of the complex plane. Although there has been considerable progress made in determining the properties of CFSs and complex fuzzy logic, there has yet to be any practical application of this concept. We present the adaptive neurocomplex-fuzzy-inferential system (ANCFIS), which is the first neurofuzzy system architecture to implement complex fuzzy rules (and, in particular, the signature property of rule interference). We have applied this neurofuzzy system to the domain of time-series forecasting, which is an important machine-learning problem. We find that ANCFIS performs well in one synthetic and five real-world forecasting problems and is also very parsimonious. Experimental comparisons show that ANCFIS is comparable with existing approaches on our five datasets. This work demonstrates the utility of complex fuzzy logic on real-world problems. [ABSTRACT FROM AUTHOR]

Layout hotspot detection is of great importance in the physical verification flow. Deep neural network models have been applied to hotspot detection and achieved great successes. The layouts can be viewed as binary images. The binarized neural network (BNN) can thus be suitable for the hotspot detection problem. In this article, we propose a new deep learning architecture based on BNNs to speed up the neural networks in hotspot detection. A new binarized residual neural network is carefully designed for hotspot detection. Experimental results on ICCAD 2012 and 2019 benchmarks show that our architecture outperforms previous hotspot detectors in detecting accuracy and has an 8 × speedup over the best deep learning-based solution. Since the BNN-based model is quite computationally efficient, a good tradeoff can be achieved between the efficiency and performance of the hotspot detector by applying ensemble learning approaches. Experimental results show that the ensemble models achieve better hotspot detection performance than the original with acceptable speed loss. [ABSTRACT FROM AUTHOR]

Objective: Enable accurate estimation of vertical average loading rate (VALR) in runners with one or more wearable inertial measurement units (IMUs). Methods: A subject-independent convolutional neural network (CNN) model was developed to estimate VALR from wearable IMUs. Fifteen runners wore IMUs at the trunk, pelvis, thigh, shank, and foot and ran on an instrumented treadmill for combinations of the following conditions: foot-strike (forefoot, mid-foot, rear-foot), step rate (90% to 110% of baseline), running speed (2.4 m/s and 2.8 m/s) and footwear (standard and minimalist running shoes). Thirty-one IMU placement configurations with combinations of one to five IMUs were evaluated. VALR estimations from the wearable IMUs were compared with force-plate VALR measurements. Results: VALR estimations via the subject-independent CNN model with a single shank-worn IMU were highly correlated (ρ = 0.94) with force-plate VALR measurements and were substantially higher than previously reported peak tibial acceleration correlations with force-plate VALR measurements from shank-worn accelerometers (ρ = 0.44–0.66). Correlation results from the CNN model for a single IMU placed at the foot, pelvis, trunk, and thigh were ρ = 0.91, 0.76, 0.69, and 0.65, respectively. There was no improvement in accuracy from the shank-worn IMU when adding 1–4 additional IMUs from the trunk, pelvis, thigh, or foot. Conclusion: The proposed subject-independent CNN model with a single shank-worn IMU provides more accurate estimation of VALR than previous wearable sensing approaches. Significance: This could enable runners to more accurately assess impact loading rates and potentially provide insights into running-related injury risk and prevention. [ABSTRACT FROM AUTHOR]

An efficient and precise liver extraction from computed tomography (CT) images is a crucial step for computer-aided hepatic diseases diagnosis and treatment. Considering the possible risk to patient's health due to X-ray radiation of repetitive CT examination, low-dose CT (LDCT) is an effective solution for medical imaging. However, inhomogeneous appearances and indistinct boundaries due to additional noise and streaks artifacts in LDCT images often make it a challenging task. This study aims to extract a liver model from LDCT images for facilitating medical expert in surgical planning and post-operative assessment along with low radiation risk to the patient. Our method carried out liver extraction by employing residual convolutional neural networks (LER-CN), which is further refined by noise removal and structure preservation components. After patch-based training, our LER-CN shows a competitive performance relative to state-of-the-art methods for both clinical and publicly available MICCAI Sliver07 datasets. We have proposed training and learning algorithms for LER-CN based on back propagation gradient descent. We have evaluated our method on 150 abdominal CT scans for liver extraction. LER-CN achieves dice similarity coefficient up to 96.5 $\pm \text{1.8}\%$ , decreased volumetric overlap error up to 4.30 $\pm \text{0.58}\%$ , and average symmetric surface distance less than 1.4 $\pm \text{0.5mm}$. These findings have shown that LER-CN is a favorable method for medical applications with high efficiency allowing low radiation risk to patients. [ABSTRACT FROM AUTHOR]

Fall detection is an important public healthcare problem. Timely detection could enable instant delivery of medical service to the injured. A popular nonintrusive solution for fall detection is based on videos obtained through ambient camera, and the corresponding methods usually require a large dataset to train a classifier and are inclined to be influenced by the image quality. However, it is hard to collect fall data and instead simulated falls are recorded to construct the training dataset, which is restricted to limited quantity. To address these problems, a three-dimensional convolutional neural network (3-D CNN) based method for fall detection is developed, which only uses video kinematic data to train an automatic feature extractor and could circumvent the requirement for large fall dataset of deep learning solution. 2-D CNN could only encode spatial information, and the employed 3-D convolution could extract motion feature from temporal sequence, which is important for fall detection. To further locate the region of interest in each frame, a long short-term memory (LSTM) based spatial visual attention scheme is incorporated. Sports dataset Sports-1 M with no fall examples is employed to train the 3-D CNN, which is then combined with LSTM to train a classifier with fall dataset. Experiments have verified the proposed scheme on fall detection benchmark with high accuracy as 100%. Superior performance has also been obtained on other activity databases. [ABSTRACT FROM AUTHOR]

Automatic recognition of an image’s style is important for many applications, including artwork analysis, photo organization, and image retrieval. Traditional convolution neural network (CNN) approach uses only object features for image style recognition. This approach may not be optimal, because the same object in two images may have different styles. We propose a CNN architecture with two pathways extracting object features and texture features, respectively. The object pathway represents the standard CNN architecture and the texture pathway intermixes the object pathway by outputting the gram matrices of intermediate features in the object pathway. The two pathways are jointly trained. In experiments, two deep CNNs, AlexNet and VGG-19, pretrained on the ImageNet classification data set are fine-tuned for this task. For any model, the two-pathway architecture performs much better than individual pathways, which indicates that the two pathways contain complementary information of an image’s style. In particular, the model based on VGG-19 achieves the state-of-the-art results on three benchmark data sets, WikiPaintings, Flickr Style, and AVA Style. [ABSTRACT FROM AUTHOR]

Answer selection in community question answering (cQA) is a challenging task in natural language processing. The difficulty lies in that it not only needs the consideration of semantic matching between question answer pairs but also requires a serious modeling of contextual factors. In this letter, we propose an attentive deep neural network architecture so as to learn the deterministic information for answer selection. The architecture can support various input formats through the organization of convolutional neural networks, attention-based long short-term memory, and conditional random fields. Experiments are carried out on the SemEval-2015 cQA dataset. We attain 58.35% on macroaveraged \mathrmF_1, which outperforms the Top-1 system in the shared task by 1.16% and improves the state-of-the-art deep-neural-network-based method by 2.21%. [ABSTRACT FROM PUBLISHER]

We propose a novel segmentation approach based on deep 3D convolutional encoder networks with shortcut connections and apply it to the segmentation of multiple sclerosis (MS) lesions in magnetic resonance images. Our model is a neural network that consists of two interconnected pathways, a convolutional pathway, which learns increasingly more abstract and higher-level image features, and a deconvolutional pathway, which predicts the final segmentation at the voxel level. The joint training of the feature extraction and prediction pathways allows for the automatic learning of features at different scales that are optimized for accuracy for any given combination of image types and segmentation task. In addition, shortcut connections between the two pathways allow high- and low-level features to be integrated, which enables the segmentation of lesions across a wide range of sizes. We have evaluated our method on two publicly available data sets (MICCAI 2008 and ISBI 2015 challenges) with the results showing that our method performs comparably to the top-ranked state-of-the-art methods, even when only relatively small data sets are available for training. In addition, we have compared our method with five freely available and widely used MS lesion segmentation methods (EMS, LST-LPA, LST-LGA, Lesion-TOADS, and SLS) on a large data set from an MS clinical trial. The results show that our method consistently outperforms these other methods across a wide range of lesion sizes. [ABSTRACT FROM AUTHOR]