Search

Fault diagnosis is vital for modern industry, and an increasing number of intelligent methods have been proposed for the fault diagnosis. However, most of the studies focus on distinguishing different fault patterns while ignoring fault deterioration. In this paper, a new hierarchical convolutional neural network (HCNN) is proposed as the two-level hierarchical diagnosis network, and it has two characteristics: 1) the fault pattern and fault severity are modeled as one hierarchical structure and 2) the fault pattern and fault severity can be estimated at the same time. Based on these, a new structure of HCNN is designed, which has two classifiers. Then, a two-stage training method is developed for HCNN to train these two classifiers at once training. The proposed HCNN is conducted on three case studies and has achieved state-of-the-art results. The results show that HCNN outperforms traditional two-layer hierarchical fault diagnosis network, and other machine learning and deep learning methods. [ABSTRACT FROM AUTHOR]

Bioimage classification is increasingly becoming more important in many biological studies including those that require accurate cell phenotype recognition, subcellular localization, and histopathological classification. In this paper, we present a new General Purpose (GenP) bioimage classification method that can be applied to a large range of classification problems. The GenP system we propose is an ensemble that combines multiple texture features (both handcrafted and learned descriptors) for superior and generalizable discriminative power. Our ensemble obtains a boosting of performance by combining local features, dense sampling features, and deep learning features. Each descriptor is used to train a different Support Vector Machine that is then combined by sum rule. We evaluate our method on a diverse set of bioimage classification tasks each represented by a benchmark database, including some of those available in the IICBU 2008 database. Each bioimage classification task represents a typical subcellular, cellular, and tissue level classification problem. Our evaluation on these datasets demonstrates that the proposed GenP bioimage ensemble obtains state-of-the-art performance without any ad-hoc dataset tuning of the parameters (thereby avoiding any risk of overfitting/overtraining). To reproduce the experiments reported in this paper, the MATLAB code of all the descriptors is available at https://github.com/LorisNanni and https://www.dropbox.com/s/bguw035yrqz0pwp/ElencoCode.docx?dl=0. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) classification is an active and important research task driven by many practical applications. To leverage deep learning models especially convolutional neural networks (CNNs) for HSI classification, this paper proposes a simple yet effective method to extract hierarchical deep spatial feature for HSI classification by exploring the power of off-the-shelf CNN models, without any additional retraining or fine-tuning on the target data set. To obtain better classification accuracy, we further propose a unified metric learning-based framework to alternately learn discriminative spectral–spatial features, which have better representation capability and train support vector machine (SVM) classifiers. To this end, we design a new objective function that explicitly embeds a metric learning regularization term into SVM training. The metric learning regularization term is used to learn a powerful spectral–spatial feature representation by fusing spectral feature and deep spatial feature, which has small intraclass scatter but big between class separation. By transforming HSI data into new spectral–spatial feature space through CNN and metric learning, we can pull the pixels from the same class closer, while pushing the different class pixels farther away. In the experiments, we comprehensively evaluate the proposed method on three commonly used HSI benchmark data sets. State-of-the-art results are achieved when compared with the existing HSI classification methods. [ABSTRACT FROM AUTHOR]

Ataxic gait monitoring and assessment of neurological disorders belong to important multidisciplinary areas that are supported by digital signal processing methods and machine learning tools. This paper presents the possibility of using accelerometric data to optimise deep learning convolutional neural network systems to distinguish between ataxic and normal gait. The experimental dataset includes 860 signal segments of 16 ataxic patients and 19 individuals from the control set with the mean age of 38.6 and 39.6 years, respectively. The proposed methodology is based upon the analysis of frequency components of accelerometric signals simultaneously recorded at specific body positions with a sampling frequency of 60 Hz. The deep learning system uses all of the frequency components in a range of $\langle {0},{30} \rangle $ Hz. Our classification results are compared with those obtained by standard methods, which include the support vector machine, Bayesian methods, and the two-layer neural network with features estimated as the relative power in selected frequency bands. Our results show that the appropriate selection of sensor positions can increase the accuracy from 81.2% for the foot position to 91.7% for the spine position. Combining the input data and the deep learning methodology with five layers increased the accuracy to 95.8%. Our methodology suggests that artificial intelligence methods and deep learning are efficient methods in the assessment of motion disorders and they have a wide range of further applications. [ABSTRACT FROM AUTHOR]

Promptly discovering unknown network attacks is critical for reducing the risk of major loss imposed on organizations and information infrastructure. This paper aims at developing an intelligent intrusion detection system capable of classifying known attacks as well as inferring unknown ones. To achieve this, we formulate the problem of fine-grained known/unknown intrusion detection as a two-stage minimization problem, where the first stage is to seek a score measure for minimizing the empirical risk of misclassifying the known attacks, while the second stage is to find another score measure for minimizing the identification risk of inferring unknown attacks. The hierarchical nature of problem formulation allows us to employ the class conditioned auto-encoders to construct a hierarchical intrusion detection framework. Since the reconstruction errors of unknown attacks are generally higher than that of the known attacks, we further employ extreme value theory in the second stage to model the distribution of reconstruction errors for differentiating known/unknown attack. To further reduce the false positive rate, we add a benign clustering module for learning the multimodal distribution of benign traffic. We conduct an experiment on two widely used datasets for assessing intrusion detection. The results show that the proposed method improves the detection rate of unknown attacks while keeping a low false positive rate. [ABSTRACT FROM AUTHOR]

Deep learning based land cover classification algorithms have recently been proposed in the literature. In hyperspectral images (HSIs), they face the challenges of large dimensionality, spatial variability of spectral signatures, and scarcity of labeled data. In this paper, we propose an end-to-end deep learning architecture that extracts band specific spectral-spatial features and performs land cover classification. The architecture has fewer independent connection weights and thus requires fewer training samples. The method is found to outperform the highest reported accuracies on popular HSI data sets. [ABSTRACT FROM PUBLISHER]

Harnessing the substantial bandwidth available at millimeter wave (mmWave) carrier frequencies has proved to be beneficial to accommodate a large number of users with increased data rates. However, owing to the high propagation losses observed at mmWave frequencies, directional transmission has to be employed. This necessitates efficient beam-alignment for a successful transmission. Achieving perfect beam-alignment is however challenging, especially in the scenarios when there is a rapid movement of vehicles associated with ever-changing traffic density, which is governed by the topology of roads as well as the time of the day. Therefore, in this paper, we take the approach of fingerprint based beam-alignment, where a set of beam pairs constitute the fingerprint of a given location. Furthermore, given the time-varying traffic density, we propose a multi-fingerprint based database for a given location, where the base station (BS) intelligently adapts the fingerprints with the aid of learning. Additionally, we propose multi-functional beam transmission as an application of our proposed design, where the beam-pairs that satisfy the required received signal strength (RSS) participate in increasing the spectral efficiency or improving the end-to-end performance in some other way. Explicitly, the BS leverages the plurality of beam-pairs to attain both multiplexing and diversity gains. Furthermore, if the plurality of beam-pairs is higher than the number of RF chains, the BS may also employ beam-index modulation to further improve the spectral efficiency. We demonstrate that having multiple fingerprint-based beam-alignment provides superior performance than that of the single fingerprint based beam-alignment. Furthermore, we show that our learning-aided multiple fingerprint design provides a better fidelity compared to that of the benchmark scheme also employing multiple fingerprint but dispensing with learning. Additionally, our reduced-search based learning-aided beam-alignment design performs similarly to beam-sweeping based beam-alignment, even though an exhaustive beam-search is carried out by the latter. More explicitly, our design is capable of maintaining the target performance in dense vehicular environments, while both single fingerprint and line-of-sight (LOS) based beam-alignment suffer from blockages. [ABSTRACT FROM AUTHOR]

Recently, many researchers have been working on improving other traditional machine learning algorithms (besides deep learning) using high-performance hardware such as Graphics Processing Units (GPUs). The recent success of machine learning is not only due to more effective algorithms, but also more efficient systems and implementations. In this paper, we propose a novel and efficient solution to multi-class SVMs with probabilistic output (MP-SVMs) accelerated by GPUs. MP-SVMs are an important technique for many pattern recognition applications. However, MP-SVMs are very time-consuming to use, because using an MP-SVM classifier requires training many binary SVMs and performing probability estimation by combining results of all the binary SVMs. GPUs have much higher computation capability than CPUs and are potentially excellent hardware to accelerate MP-SVMs. Still, two key challenges for efficient GPU accelerations for MP-SVM are: (i) many kernel values are repeatedly computed as a binary SVM classifier is trained iteratively, resulting in repeated accesses to the high latency GPU memory; (ii) performing training or estimating probability in a highly parallel way requires a much larger memory footprint than the GPU memory. To overcome the challenges, we propose a solution called GMP-SVM which exploits two-level (i.e., binary SVM level and MP-SVM level) optimization for training MP-SVMs and high parallelism for estimating probability. GMP-SVM reduces high latency memory accesses and memory consumption through batch processing, kernel value reusing and sharing, and support vector sharing. Experimental results show that GMP-SVM outperforms the GPU baseline by two to five times, and LibSVM with OpenMP by an order of magnitude. Also, GMP-SVM produces the same SVM classifier as LibSVM. [ABSTRACT FROM AUTHOR]

In a weakly-supervised scenario object detectors need to be trained using image-level annotation alone. Since bounding-box-level ground truth is not available, most of the solutions proposed so far are based on an iterative, Multiple Instance Learning framework in which the current classifier is used to select the highest-confidence boxes in each image, which are treated as pseudo-ground truth in the next training iteration. However, the errors of an immature classifier can make the process drift, usually introducing many of false positives in the training dataset. To alleviate this problem, we propose in this paper a training protocol based on the self-paced learning paradigm. The main idea is to iteratively select a subset of images and boxes that are the most reliable, and use them for training. While in the past few years similar strategies have been adopted for SVMs and other classifiers, we are the first showing that a self-paced approach can be used with deep-network-based classifiers in an end-to-end training pipeline. The method we propose is built on the fully-supervised Fast-RCNN architecture and can be applied to similar architectures which represent the input image as a bag of boxes. We show state-of-the-art results on Pascal VOC 2007, Pascal VOC 2010 and ILSVRC 2013. On ILSVRC 2013 our results based on a low-capacity AlexNet network outperform even those weakly-supervised approaches which are based on much higher-capacity networks. [ABSTRACT FROM AUTHOR]

In this paper, we propose deformable deep convolutional neural networks for generic object detection. This new deep learning object detection framework has innovations in multiple aspects. In the proposed new deep architecture, a new deformation constrained pooling (def-pooling) layer models the deformation of object parts with geometric constraint and penalty. A new pre-training strategy is proposed to learn feature representations more suitable for the object detection task and with good generalization capability. By changing the net structures, training strategies, adding and removing some key components in the detection pipeline, a set of models with large diversity are obtained, which significantly improves the effectiveness of model averaging. The proposed approach improves the mean averaged precision obtained by RCNN [1] , which was the state-of-the-art, from $31$ to $50.3$ percent on the ILSVRC2014 detection test set. It also outperforms the winner of ILSVRC2014, GoogLeNet, by 6.1 percent. Detailed component-wise analysis is also provided through extensive experimental evaluation, which provides a global view for people to understand the deep learning object detection pipeline. [ABSTRACT FROM PUBLISHER]

Fusing the advantages of multiple acoustic features is important for the robustness of voice activity detection (VAD). Recently, the machine-learning-based VADs have shown a superiority to traditional VADs on multiple feature fusion tasks. However, existing machine-learning-based VADs only utilize shallow models, which cannot explore the underlying manifold of the features. In this paper, we propose to fuse multiple features via a deep model, called deep belief network (DBN). DBN is a powerful hierarchical generative model for feature extraction. It can describe highly variant functions and discover the manifold of the features. We take the multiple serially-concatenated features as the input layer of DBN, and then extract a new feature by transferring these features through multiple nonlinear hidden layers. Finally, we predict the class of the new feature by a linear classifier. We further analyze that even a single-hidden-layer-based belief network is as powerful as the state-of-the-art models in the machine-learning-based VADs. In our empirical comparison, ten common features are used for performance analysis. Extensive experimental results on the AURORA2 corpus show that the DBN-based VAD not only outperforms eleven referenced VADs, but also can meet the real-time detection demand of VAD. The results also show that the DBN-based VAD can fuse the advantages of multiple features effectively. [ABSTRACT FROM AUTHOR]

Ground-penetrating radar (GPR) target detection and classification is a challenging task. Here, we consider online dictionary learning (DL) methods to obtain sparse representations (SR) of the GPR data to enhance feature extraction for target classification via support vector machines. Online methods are preferred because traditional batch DL like K-times singular value decomposition (K-SVD) is not scalable to high-dimensional training sets and infeasible for real-time operation. We also develop Drop-Off MINi-batch Online Dictionary Learning (DOMINODL), which exploits the fact that a lot of the training data may be correlated. The DOMINODL algorithm iteratively considers elements of the training set in small batches and drops off samples which become less relevant. For the case of abandoned anti-personnel landmines classification, we compare the performance of K-SVD with three online algorithms: classical online dictionary learning (ODL), its correlation-based variant, and DOMINODL. Our experiments with real data from L-band GPR show that online DL methods reduce learning time by 36%–93% and increase mine detection by 4%–28% over K-SVD. Our DOMINODL is the fastest and retains similar classification performance as the other two online DL approaches. We use a Kolmogorov–Smirnoff test distance and the Dvoretzky–Kiefer–Wolfowitz inequality for the selection of DL input parameters leading to enhanced classification results. To further compare with the state-of-the-art classification approaches, we evaluate a convolutional neural network (CNN) classifier, which performs worse than the proposed approach. Moreover, when the acquired samples are randomly reduced by 25%, 50%, and 75%, sparse decomposition-based classification with DL remains robust while the CNN accuracy is drastically compromised. [ABSTRACT FROM AUTHOR]

The performance of an existing Devanagari script (DS) input-based P300 speller with conventional machine learning techniques suffers from low information transfer rate (ITR). This occurs due to its required large size of display, i.e., 8 × 8 row-column (RC) paradigm which exhibits issues like crowding effect, adjacency, fatigue, task difficulty, and required large number of trials for character recognition. For P300 detection, deep learning algorithms have shown the state of art performance compared to the conventional machine learning algorithms in the recent past. Therefore, authors have been motivated to develop a deep learning architecture for DS-based P300 speller which can detect the target characters more accurately and in less number of trials. For this, two proven deep learning algorithms, stacked autoencoder (SAE) and deep convolution neural network (DCNN) have been adopted. For further bettering their performances, batch normalization and innovative double batch training is included here to achieve accelerated training and alleviate the problem of overfitting. Additionally, a leaky ReLU activation function has also been used in DCNN to overcome dying ReLU problem. The experiments have been performed on self-generated dataset of 20 Devanagari words with 79 characters acquired from 10 subjects using 16 channel actiCAP Xpress EEG recorder. The experimental results illustrated that the proposed DCNN is able to detect 88.22% correct targets in just three trials. Moreover, it also provides ITR of 20.58 bits per minutes, which is significantly higher than existing techniques. [ABSTRACT FROM AUTHOR]

Highlight detection from videos has been widely studied due to the fast growth of video contents. However, most existing approaches to highlight detection, either handcraft feature based or deep learning based, heavily rely on human-curated training data, which is very expensive to obtain and, thus, hinders the scalability to large datasets and unlabeled video categories. We observe that the largely available Web images can be applied as a weak supervision for highlight detection. For example, the top-ranked images in reference to the query “skiing” returned by a search engine may contain considerable positive samples of “skiing” highlights. Motivated by this observation, we propose a novel triplet deep ranking approach to video highlight detection using Web images as a weak supervision. The approach handles the relative preference of highlight scores between highlighting frames, nonhighlighting frames, and Web images by the triplet ranking constraints. Our approach can iteratively train two interdependent deep models (i.e., a triplet highlight model and a pairwise noise model) to deal with the noisy Web images in a single framework. We train the two models with relative preferences to generalize the capability regardless of the categories of training data. Therefore, our approach is fully category independent and exploits weakly supervised Web images. We evaluate our approach on two challenging datasets and achieve impressive results compared with the state-of-the-art pairwise ranking support vector machines, a robust recurrent autoencoder, and spatial deep convolution neural networks. We also empirically verify through cross-dataset evaluation that our category-independent model is fairly generalizable even if two different datasets do not share exactly the same categories. [ABSTRACT FROM AUTHOR]

A combination of a convolutional neural network, which belongs to the deep learning research field, and support vector machines is presented as an efficient automatic target recognition system. Additional training methods that incorporate prior knowledge to the classifier and further improve its robustness against imaging errors and target variations are also presented. These methods generate artificial training data by elastic distortion and affine transformations that represent typical examples of image errors, like a changing range scale dependent on the depression angle or an incorrectly estimated aspect angle. With these examples presented to the classifier during the training, the system should become invariant against these variations and thus more robust. For the classification, the spotlight synthetic aperture radar images of the moving and stationary target acquisition and recognition database are used. Results are shown for the ten class database with a forced decision classification as well as with rejection class. [ABSTRACT FROM AUTHOR]