Search

Conventional machine learning algorithms suffer the problem that the model trained on existing data fails to generalize well to the data sampled from other distributions. To tackle this issue, unsupervised domain adaptation (UDA) transfers the knowledge learned from a well-labeled source domain to a different but related target domain where labeled data is unavailable. The majority of existing UDA methods assume that data from the source domain and the target domain are available and complete during training. Thus, the divergence between the two domains can be formulated and minimized. In this paper, we consider a more practical yet challenging UDA setting where either the source domain data or the target domain data are unknown. Conventional UDA methods would fail this setting since the domain divergence is agnostic due to the absence of the source data or the target data. Technically, we investigate UDA from a novel view—adversarial attack—and tackle the divergence-agnostic adaptive learning problem in a unified framework. Specifically, we first report the motivation of our approach by investigating the inherent relationship between UDA and adversarial attacks. Then we elaborately design adversarial examples to attack the training model and harness these adversarial examples. We argue that the generalization ability of the model would be significantly improved if it can defend against our attack, so as to improve the performance on the target domain. Theoretically, we analyze the generalization bound for our method based on domain adaptation theories. Extensive experimental results on multiple UDA benchmarks under conventional, source-absent and target-absent UDA settings verify that our method is able to achieve a favorable performance compared with previous ones. Notably, this work extends the scope of both domain adaptation and adversarial attack, and expected to inspire more ideas in the community. [ABSTRACT FROM AUTHOR]

Classification of 3D objects – the selection of a category in which each object belongs – is of great interest in the field of machine learning. Numerous researchers use deep neural networks to address this problem, altering the network architecture and representation of the 3D shape used as an input. To investigate the effectiveness of their approaches, we conduct an extensive survey of existing methods and identify common ideas by which we categorize them into a taxonomy. Second, we evaluate 11 selected classification networks on two 3D object datasets, extending the evaluation to a larger dataset on which most of the selected approaches have not been tested yet. For this, we provide a framework for converting shapes from common 3D mesh formats into formats native to each network, and for training and evaluating different classification approaches on this data. Despite being partially unable to reach the accuracies reported in the original papers, we compare the relative performance of the approaches as well as their performance when changing datasets as the only variable to provide valuable insights into performance on different kinds of data. We make our code available to simplify running training experiments with multiple neural networks with different prerequisites. [ABSTRACT FROM AUTHOR]

Many learning algorithms such as kernel machines, nearest neighbors, clustering, or anomaly detection, are based on distances or similarities. Before similarities are used for training an actual machine learning model, we would like to verify that they are bound to meaningful patterns in the data. In this paper, we propose to make similarities interpretable by augmenting them with an explanation. We develop BiLRP, a scalable and theoretically founded method to systematically decompose the output of an already trained deep similarity model on pairs of input features. Our method can be expressed as a composition of LRP explanations, which were shown in previous works to scale to highly nonlinear models. Through an extensive set of experiments, we demonstrate that BiLRP robustly explains complex similarity models, e.g., built on VGG-16 deep neural network features. Additionally, we apply our method to an open problem in digital humanities: detailed assessment of similarity between historical documents, such as astronomical tables. Here again, BiLRP provides insight and brings verifiability into a highly engineered and problem-specific similarity model. [ABSTRACT FROM AUTHOR]

Olfaction transduction mechanism is triggered by the binding of odorants to the specific olfactory receptors (OR's) present in the nasal cavity. Different odorants stimulate different OR's due to the difference in shape, physical and chemical properties. In this paper, a deep neural network architecture DeepOlf, based on molecular features and fingerprints of odorants and ORs, to predict whether a chemical compound is a potential odorant or not along with its interacting OR is proposed. Odorant identification and Odorant-OR interaction were modeled as a binary classification through multiple classifiers. The evaluation of these classifier's performance showed that the deep-neural network framework not only fits data with better accuracy in comparison to other classical methods (SVM, RF, k-NN) but also able to predict odorant-OR interactions more accurately. To our knowledge, this study is the first realization of deep learning ideas for the problem of odorant and interacting OR prediction. The accuracy of DeepOlf was found to be 94.83 and 99.92 percent for the prediction of odorants and Odorant- OR interactions respectively. Comparison of DeepOlf prediction with the existing SVM based prediction server, ODORactor, showed that better performance can be achieved with the proposed deep learning approach. The DeepOlf tool can be accessed at https://bioserver.iiita.ac.in/deepolf/. [ABSTRACT FROM AUTHOR]

Transportation services play an increasingly significant role for people’s daily lives and bring a lot of benefits to individuals and economic development. The randomness and volatility of traffic flows, however, constrains the effective provision of transportation services to a certain extent. Precise traffic flow forecasting becomes the key and primary task to realize the stability of intelligent transport systems and ensure efficient scheduling of traffic. This paper investigates the application of an ensemble approach based on deep belief networks for short-term traffic flow forecasting. Traffic flow data, collected from the real world, is decomposed into several Intrinsic Mode Functions (IMFs) and a residue with EEMD (Ensemble Empirical Mode Decomposition). Then, for each component, the essential feature subset is extracted by the mRMR (minimum Redundancy Maximum Relevance Feature Selection) method considering weather conditions and day properties. Furthermore, each component is trained by DBN (Deep belief networks) and their forecasting results are summed up as the output of the ensemble model at last. Results indicate that the proposed approach achieves significant performance improvement over the single DBN and other selected methods. [ABSTRACT FROM AUTHOR]

Seizure prediction for drug-refractory epilepsy patients can improve their quality of life, reduce their anxiety, and help them take the necessary precautions. Nowadays, numerous deep learning algorithms have been proposed to predict seizure onset and obtain better performance than traditional machine learning methods. However, these methods require a large set of parameters and large hardware resources; they also have high energy consumption. Therefore, these methods cannot be implemented on compact, low-power wearable, or implantable medical devices. The devices should operate on a real-time basis to continually inform the epileptic patients. In this paper, we describe energy-efficient and hardware-friendly methods to predict the epileptic seizures. A model of only 45 kB was obtained by the neural architecture search and was evaluated across three datasets. The overall sensitivity, false prediction rate, and area under receiver operating characteristic curve were 99.81%, 0.005/h, 1 and 93.48%, 0.063/h, 0.977 and 85.19%, 0.116/h, 0.933, respectively, for three public datasets. This model was further reduced with several methods for model compression. The performances for the model sizes less than 50 kB for scalp EEG data and less than 10 kB for intracranial EEG data outperformed all the other models of similar model sizes. In particular, the energy consumption estimation was less than 10 mJ per inference for scalp EEG signal and less than 0.5 mJ per inference for intracranial EEG signal, which meet the requirements for low-power wearable and implantable devices, respectively. [ABSTRACT FROM AUTHOR]

Humans have rich experience applying linear models and logical thinking, but only experts understand the behaviour of non-linear systems. However, the deep neural network (DNN) implementation of text non-linear systems outperforms optimal linear models. Therefore, the forward DNN (the pattern recognition system in this paper) attracts attention to the necessity of interpreting the results obtained by DNN. To preserve the high performance of DNN, we focus on a post-hoc explanation; this approach means building an explainable model for the decision obtained by the black box. To avoid the interpretation of a set of millions of non-linear functions, we divide DNN into two parts: the feature extractor and the classifier. Following that, we argue for a specific interpretation of each of them. While for classifiers, we have several suitable explainable models (and we decided on the fuzzy logical function), we believe that feature interpretation is a creative scientific activity corresponding to the usual research. The paper presents a tool to help researchers and users understand extracted features not necessarily known in the specific application domain. Explaining the new features offers a way to learn from computers.

The downlink channel estimation requires a huge pilot overhead in the reconfigurable intelligent surface (RIS) assisted communication system. By exploiting the powerful learning ability of the neural network, the machine learning (ML) technique can be used to estimate the high-dimensional channel from a few received pilot signals at the user. However, since the training dataset collected by the single user only contains the information of part of the channel scenarios of a cell, the neural network trained by the single user is not able to work when the user moves from one channel scenario to another. To solve this challenge, we propose to leverage the distributed machine learning (DML) technique to enable the reliable downlink channel estimation. Specifically, we firstly build a downlink channel estimation neural network shared by all users, which can be collaboratively trained by the BS and the users with the help of the DML technique. Then, we further propose a hierarchical neural network architecture to improve the channel estimation accuracy, which can extract different channel features for different channel scenarios. Simulation results show that compared with the neural network trained by the single user, the proposed DML based neural networks can achieve better estimation performance with the reduced pilot overhead for all users from different scenarios. [ABSTRACT FROM AUTHOR]

Micro-expression is a spontaneous expression that occurs when a person tries to mask his or her inner emotion, and can neither be forged nor suppressed. It is a kind of short-duration, low-intensity, and usually local-motion facial expression. However, owing to these characteristics of micro-expression, it is difficult to obtain micro-expression data, which is the bottleneck of applying deep learning methods to micro-expression recognition. In addition, micro-expression is still a type of expression, and it can also be encoded by the facial action coding system. Therefore, there is a certain correlation between action unit recognition and micro-expression recognition. Addressing those, we propose a novel knowledge transfer technique distills and transfers knowledge from action unit for micro-expression recognition, where knowledge from a pre-trained deep teacher neural network is distilled and transferred to a shallow student neural network. Specifically, a teacher-student correlative framework is designed with a novel objective function. And features extracted from the teacher network is used as prior knowledge to guide the student part to efficiently learning from the target micro-expression dataset. Experiments are conducted on four available published micro-expression datasets (SMIC2, CASME, CASME II, and SAMM). The experimental results show that our model outperforms the state-of-the-art systems. [ABSTRACT FROM AUTHOR]

An end-to-end supervised learning method is proposed for fault detection in the electric grid using Big Data from multiple Phasor Measurement Units (PMUs). The approach consists of preprocessing steps aimed at reducing data noise and dimensionality, followed by utilization of six classification models considered for detecting faults. Three of the models were variants of Convolutional Neural Network (CNN) architectures that consider a single type of measurement (voltage, current or frequency) at all PMUs or all types together also at all PMUs. CNN based models were compared to traditional methods of Logistic Regression (LR), Multi-layer Perceptron (MLP) and Support Vector Machine (SVM). Evaluation was conducted on two-year data measured by PMUs at 37 locations in a large electric grid. The response variable for classification were extracted from the grid-wide outage event log. Experiments show that CNN-based models outperformed traditional methods on one year out-of-sample outage detection over the entire grid. [ABSTRACT FROM AUTHOR]

In this article, interferencequality assessment is of great significance to reflect the communication environment and improve speech communication performance. However, most traditional assessment approaches aimed at the degraded speech produced in the communication telephone network, but lacking of methods in extreme communication environment with ultralow SNR. Therefore, in this article, we proposed a convolutional neural network (CNN) model evaluation method based on Log-Mel spectrogram to evaluate interfered speech quality. In this method, the Mel frequency cepstrum coefficients of interference speech are converted into images, which are used as input of CNN. In order to verify the performance of this method, we collected a speech dataset in real interfered communication scenarios and finished manual annotation. Experiments are carried out on this dataset to evaluate the interference speech, and the performance of this method is compared with that of machine learning evaluation method under different features. Experimental results show that the proposed method gives the better evaluation accuracy. Compared with the previous machine learning methods, the accuracy is improved by 12.5% from 75% to 87.5%. [ABSTRACT FROM AUTHOR]

As the circuit feature size continuously shrinks down, hotspot detection has become a more challenging problem in modern design for manufacturability flows. Developed deep learning techniques have recently shown their superiorities on hotspot detection tasks. However, existing hotspot detectors can only handle defect detection from one small layout clip each time, thus, may be very time-consuming when dealing with a large full-chip layout. In this article, we develop a new end-to-end framework that can detect multiple hotspots in a large region at a time and promise a better hotspot detection performance. We design a joint auto-encoder and inception module for efficient feature extraction. A two-stage classification and regression framework is designed to detect hotspot with progressive accurate localization, which provides a promising performance improvement. Experimental results show that our framework enables a significant speed improvement over existing methods with higher accuracy and fewer false alarms. [ABSTRACT FROM AUTHOR]

In this brief, we propose an audio feature extractor, based on a time delay neural network (TDNN), for ultra-low power keyword spotting (KWS). Conventionally, mel-frequency cepstrum coefficients (MFCCs) are widely used as features for KWS. However, an analog-to-digital converter (ADC) with high precision and high sampling rate is required for computing MFCCs, which consumes large amount of power. In our proposed feature extractor, on the other hand, input audio signals are band-pass filtered, and then the filtered signals are processed in TDNN in the analog domain, which can substitute a high precision ADC with simple clocked comparators. Simulation results show that the power dissipation of the proposed feature extractor can be reduced by 88% compared to the conventional MFCC feature extractor. [ABSTRACT FROM AUTHOR]

Machine remaining useful life (RUL) prediction is vital in improving the reliability of industrial systems and reducing maintenance cost. Recently, long short-term memory (LSTM) based algorithms have achieved state-of-the-art performance for RUL prediction due to their strong capability of modeling sequential sensory data. In many cases, the RUL prediction algorithms are required to be deployed on edge devices to support real-time decision making, reduce the data communication cost, and preserve the data privacy. However, the powerful LSTM-based methods which have high complexity cannot be deployed to edge devices with limited computational power and memory. To solve this problem, we propose a knowledge distillation framework, entitled KDnet-RUL, to compress a complex LSTM-based method for RUL prediction. Specifically, it includes a generative adversarial network based knowledge distillation (GAN-KD) for disparate architecture knowledge transfer, a learning-during-teaching based knowledge distillation (LDT-KD) for identical architecture knowledge transfer, and a sequential distillation upon LDT-KD for complicated datasets. We leverage simple and complicated datasets to verify the effectiveness of the proposed KDnet-RUL. The results demonstrate that the proposed method significantly outperforms state-of-the-art KD methods. The compressed model with 12.8 times less weights and 46.2 times less total float point operations even achieves a comparable performance with the complex LSTM model for RUL prediction. [ABSTRACT FROM AUTHOR]

Supervised hyperspectral image (HSI) classification has been widely studied and used in many different applications. However, the performance of the supervised classifiers, including the traditional machine learning methods and the deep neural networks, is significantly affected by the inaccurate labeling of training samples, which is a common problem in HSI supervised classification. In this article, we propose a superpixel guided sample selection neural network (S3Net) framework with end-to-end training for handling noisy labels in HSI classification. It includes two stages: sample selection and sample correction. In sample selection, a sample with a small training loss has a higher probability of being the correct label and hence selected from the noisy labels for model training. In order to avoid the error propagation caused by the noisy labels, we utilize a cross-selection update strategy that exchanges selected samples between two neural networks during conventional loss backpropagation. Sample selection is a pruning process, which may cause insufficient training sample problem in HSI classification. To solve this problem, we propose the sample correction strategy to correct the noisy labels by propagating clean label information in the homogeneous regions obtained by superpixel. Experimental results on three public HSI data sets demonstrate the effectiveness of the proposed S3Net framework when handling noisy labels. [ABSTRACT FROM AUTHOR]

In unsupervised domain adaptation (UDA), a classifier for a target domain is trained with labeled source data and unlabeled target data. Existing UDA methods assume that the source data come from the same source domain (i.e., single-source scenario) or from multiple source domains, whose feature spaces have the same dimension (homogeneous) but different distributions (i.e., multihomogeneous-source scenario). However, in the real world, for a specific target domain, we probably have multiple different-dimension (heterogeneous) source domains, which do not satisfy the assumption of existing UDA methods. To remove this assumption and move forward to a realistic UDA problem, this article presents a shared-fuzzy-equivalence-relation neural network (SFERNN) for addressing the multisource heterogeneous UDA problem. The SFERNN is a five-layer neural network containing $c$ source branches and one target branch. The network structure of the SFERNN is first confirmed by a novel fuzzy relation called multisource shared fuzzy equivalence relation. Then, we optimize parameters of the SFERNN via minimizing cross-entropy loss on $c$ source branches and the distributional discrepancy between each source branch and the target branch. Experiments distributed across eight real-world datasets are conducted to validate the SFERNN. This testing regime demonstrates that the SFERNN outperforms the existing single-source heterogeneous UDA methods, especially when the target domain contains few data. [ABSTRACT FROM AUTHOR]

Feature extraction is a critical issue in many machine learning systems. A number of basic fusion operators have been proposed and studied. This article proposes an evolutionary algorithm, called evolutionary deep fusion method, for searching an optimal combination scheme of different basic fusion operators to fuse multiview features. We apply our proposed method to chemical structure recognition. Our proposed method can directly take images as inputs, and users do not need to transform images to other formats. The experimental results demonstrate that our proposed method can achieve a better performance than those designed by human experts on this real-life problem. [ABSTRACT FROM AUTHOR]