Search

The approach of dynamical system (DS) is promising for modeling robot motion, and provides a flexible means of realizing robot learning and control. Accuracy, stability, and learning speed are the three main factors to be considered when learning robot movements from human demonstrations with DS. Some approaches yield stable dynamical systems, but these may result in a poor reproduction performance, while some approaches yield good reproduction performance but are quite complex and time-consuming. In this paper, we address the accuracy-stability-speed issues simultaneously. We present a learning method named the fast and stable modeling for dynamical systems, which is based on the extreme learning machine to efficiently and accurately learn the parameters of the DS as well as to ensure the asymptotic stability at the target. We confirm the proposed approach by performing both 2-D tasks of learning handwriting motions and a set of robot experiments. [ABSTRACT FROM AUTHOR]

A general deep learning (DL) mechanism for a multiple hidden layer feed-forward neural network contains two parts, i.e., 1) an unsupervised greedy layer-wise training and 2) a supervised fine-tuning which is usually an iterative process. Although this mechanism has been demonstrated in many fields to be able to significantly improve the generalization of neural network, there is no clear evidence to show which one of the two parts plays the essential role for the generalization improvement, resulting in an argument within the DL community. Focusing on this argument, this paper proposes a new DL approach to train multilayer feed-forward neural networks. This approach uses restricted Boltzmann machine (RBM) as the layer-wise training and uses the generalized inverse of a matrix as the supervised fine-tuning. Different from the general deep training mechanism like back-propagation (BP), the proposed approach does not need to iteratively tune the weights, and therefore, has many advantages such as quick training, better generalization, and high understandability, etc. Experimentally, the proposed approach demonstrates an excellent performance in comparison with BP-based DL and the traditional training method for multilayer random weight neural networks. To a great extent, this paper demonstrates that the supervised part plays a more important role than the unsupervised part in DL, which provides some new viewpoints for exploring the essence of DL. [ABSTRACT FROM AUTHOR]

Feature selection is an optional preprocessing procedure and is frequently used to improve the classification accuracy of a machine learning algorithm by removing irrelevant and/or redundant features. However, in many real-world applications, the test cost is also required for making optimal decisions, in addition to the classification accuracy. To the best of our knowledge, thus far, few studies have been conducted on test-cost-sensitive feature selection (TCSFS). In TCSFS, the objectives are twofold: 1) to improve the classification accuracy and 2) to decrease the test cost. Therefore, in fact, it constitutes a multiobjective optimization problem. In this paper, we transformed this multiobjective optimization problem into a single-objective optimization problem by utilizing a new evaluation function and in this paper, we propose a new general wrapper framework for TCSFS. Specifically, in our proposed framework, we add a new term to the evaluation function of a wrapper feature selection method so that the test cost of measuring features is taken into account. We experimentally tested our proposed framework, using 36 classification problems from the University of California at Irvine (UCI) repository, and compared it to some other state-of-the-art feature selection frameworks. The experimental results showed that our framework allows users to select an optimal feature subset with the minimal test cost, while simultaneously maintaining a high classification accuracy. [ABSTRACT FROM AUTHOR]

The learning algorithm of an extreme learning machine (ELM) has two fundamental steps: 1) random nonlinear feature transformation and 2) least squares learning. Since the probabilistic interpretation for a sample by the least squares method follows a Gaussian distribution, there are two limitations in ELM caused by the second step: 1) it may be inaccurate to handle binary classification problems, since the output of a binary dataset has a distribution far from Gaussian and 2) it may have difficulties in dealing with nontraditional data types (such as count data, ordinal data, etc.), which also do not follow Gaussian distribution. In order to solve the above-mentioned problems, this paper proposes a generalized ELM (GELM) framework by applying the exponential family distribution (EFD) to the output layer node of ELM. It simplifies the design of ELM models for task-specific output domains with different data types. We propose a unified learning paradigm for all the models under this GELM framework with different distributions in EFD, and prove that traditional ELM is a special instance of GELM by setting the output distribution as a Gaussian distribution (GELM-Gaussian). We also prove that the training of GELM-Gaussian can be finished in one iteration, in this case, GELM-Gaussian does not slow down the training speed of traditional ELM. Besides, we propose the kernel version of GELM, which can also be concretized to different models by applying different EFDs. Experimental comparisons demonstrate that GELM can give more accurate probabilistic interpretation to binary classification and GELM has a great potential in dealing with a broader range of machine learning tasks. [ABSTRACT FROM AUTHOR]

The extreme learning machine (ELM), which was originally proposed for “generalized” single-hidden layer feedforward neural networks, provides efficient unified learning solutions for the applications of regression and classification. Although, it provides promising performance and robustness and has been used for various applications, the single-layer architecture possibly lacks the effectiveness when applied for natural signals. In order to over come this shortcoming, the following work indicates a new architecture based on multilayer network framework. The significant contribution of this paper are as follows: 1) unlike existing multilayer ELM, in which hidden nodes are obtained randomly, in this paper all hidden layers with invertible functions are calculated by pulling the network output back and putting it into hidden layers. Thus, the feature learning is enriched by additional information, which results in better performance; 2) in contrast to the existing multilayer network methods, which are usually efficient for classification applications, the proposed architecture is implemented for dimension reduction and image reconstruction; and 3) unlike other iterative learning-based deep networks (DL), the hidden layers of the proposed method are obtained via four steps. Therefore, it has much better learning efficiency than DL. Experimental results on 33 datasets indicate that, in comparison to the other existing dimension reduction techniques, the proposed method performs competitively better with fast training speeds. [ABSTRACT FROM AUTHOR]

In this paper, a novel Takagi-Sugeno–Kang (TSK) fuzzy classification system (FCS) is firstly presented for pattern classification tasks. It is distinguished by having the large margin criterion properly integrated into its objective function. In order to exploit the applicability of fuzzy systems in multiview scenarios, the proposed TSK-FCS is extended to a two-view version, called two-view TSK-FCS (TwoV-TSK-FCS), by using a collaborative learning mechanism. The adopted collaborative learning mechanism not only fully considers the independent information of each view, but also effectively discovers the correlation information hidden in the two views. Thus, the performance of TwoV-TSK-FCS can be enhanced accordingly. Comprehensive experiments on two-view synthetic and UCI datasets demonstrate the effectiveness of the proposed two-view FCS. [ABSTRACT FROM AUTHOR]

The electronic nose (E-nose), as a metal oxide semiconductor gas sensor system coupled with pattern recognition algorithms, is developed for approximating artificial olfaction functions. Ideal gas sensors should be with selectivity, reliability, and cross-sensitivity to different odors. However, a new problem is that abnormal odors (e.g., perfume, alcohol, etc.) would show strong sensor response, such that they deteriorate the usual usage of E-nose for target odor analysis. An intuitive idea is to recognize abnormal odors and remove them online. A known truth is that the kinds of abnormal odors are countless in real-world scenarios. Therefore, general pattern classification algorithms lose effect because it is expensive and unrealistic to obtain all kinds of abnormal odors data. In this paper, we propose two simple yet effective methods for abnormal odor (outlier) detection: 1) a self-expression model (SEM) with $ {l_{1}/l_{2}}$ -norm regularizer is proposed, which is trained on target odor data for coding and then a very few abnormal odor data is used as prior knowledge for threshold learning and 2) inspired by self-expression mechanism, an extreme learning machine (ELM) based self-expression (SE $^{ {2}}$ LM) is proposed, which inherits the advantages of ELM in solving a single hidden layer feed-forward neural network. Experiments on several datasets by an E-nose system fabricated in our laboratory prove that the proposed SEM and SE $^{ {2}}$ LM methods are significantly effective for real-time abnormal odor detection. [ABSTRACT FROM AUTHOR]

A system to perform video analytics is proposed using a dynamically tuned convolutional network. Videos are fetched from cloud storage, preprocessed, and a model for supporting classification is developed on these video streams using cloud-based infrastructure. A key focus in this paper is on tuning hyper-parameters associated with the deep learning algorithm used to construct the model. We further propose an automatic video object classification pipeline to validate the system. The mathematical model used to support hyper-parameter tuning improves performance of the proposed pipeline, and outcomes of various parameters on system’s performance is compared. Subsequently, the parameters that contribute toward the most optimal performance are selected for the video object classification pipeline. Our experiment-based validation reveals an accuracy and precision of 97% and 96%, respectively. The system proved to be scalable, robust, and customizable for a variety of different applications. [ABSTRACT FROM AUTHOR]

Dimension reduction (DR)-based on extreme learning machine auto-encoder (ELM-AE) has achieved many successes in recent years. By minimizing the self-reconstruction error, the ELM-AE-based DR algorithms learn the compressed representations which facilitate the subsequent classification. However, the existing ELM-AEs only consider the DR problem in an unsupervised manner and ignore the valuable supervised information when these information is available. To find discriminative features of the original data, in this paper, we propose a graph embedding-based DR framework with ELM (GDR-ELM) for DR problems. Instead of self-reconstruction, the proposed GDR-ELM reconstructs all samples according to the weights in a graph matrix containing the supervised information. Furthermore, GDR-ELM can be stacked as building blocks to construct a multilayer framework like other ELM-AEs for more complicated representation learning tasks. Experiments on various datasets demonstrate the effectiveness of the proposed GDR-ELM and its multilayer framework. [ABSTRACT FROM AUTHOR]

In the past few years, kernel-based one-class extreme learning machine (ELM) receives quite a lot of attention by researchers for offline/batch learning due to its noniterative and fast learning capability. This paper extends this concept for adaptive online learning with regularized kernel-based one-class ELM classifiers for detection of outliers, and are collectively referred to as ORK-OCELM. Two frameworks, viz., boundary and reconstruction, are presented to detect the target class in ORK-OCELM. The kernel hyperplane-based baseline one-class ELM model considers whole data in a single chunk, however, the proposed one-class classifiers are adapted in an online fashion from the stream of training samples. The performance of ORK-OCELM is evaluated on a standard benchmark as well as synthetic datasets for both types of environments, i.e., stationary and nonstationary. While evaluating on stationary datasets, these classifiers are compared against batch learning-based one-class classifiers. Similarly, while evaluating on nonstationary datasets, the comparison is done with incremental learning-based online one-class classifiers. The results indicate that the proposed classifiers yield better or similar outcomes for both. In the nonstationary dataset evaluation, adaptability of the proposed classifiers in a changing environment is also demonstrated. It is further shown that the proposed classifiers have large stream data handling capability even under limited system memory. Moreover, the proposed classifiers gain significant time improvement compared to traditional online one-class classifiers (in all aspects of training and testing). A faster learning ability of the proposed classifiers makes them more suitable for real-time anomaly detection. [ABSTRACT FROM AUTHOR]

In this paper, we address the learning problem of classifying untouched tactile instance with the help of visual modality. The proposed method is based on dictionary learning and we impose different penalty terms on coding vectors between visual and tactile modalities. Using such structured coding vectors, the visual-tactile cross-modal transfer can be achieved. A set of optimization algorithms are developed to obtain the solutions of the proposed optimization problems. After then, we can use the obtained dictionary to predict the coding vectors of the new untouched tactile samples and further determine its label. Finally, we perform extensive experimental evaluations on publicly available datasets to show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Seam carving is a content-aware image processing approach that has been successfully applied to resizing or removing objects from digital images. Seam-carved images are hard to identify from the original image, making this detection method an important and attractive research topic. Existing methods are based on steganography attacks or statistical features, and their trained and tested models are all constructed using support vector machines (SVMs). The trained models of these methods take a long time to build from a limited number of images. This paper presents a new patch-based sobel operator (PSO) method using SVM and extreme learning machines (ELMs) based on the patch analysis method with square-based features. The ELM adopts five types of neurons to obtain five prediction results, which are then combined using the jury voting scheme in order to improve the accuracy. In addition, combinatorial fusion is first used to choose two of five prediction results according to the diversity rank/score graph, and these two results are then combined to make the final decision. The PSO method in the ELM with jury voting and combinatorial fusion achieves accuracies of 70.60–98.80% and 95.58–99.36% for 10–50% seam-carved and seam-insertion images, respectively. Additionally, for the PSO method, the ELM required only 1.2% of the training time required by the SVM. In conclusion, the PSO method in the ELM is useful for constructing trained models from a large number of images. [ABSTRACT FROM AUTHOR]

Nowadays, datasets are always dynamic and patterns in them are changing. Instances with different labels are intertwined and often linearly inseparable, which bring new challenges to traditional learning algorithms. This paper proposes adaptive hyper-sphere (AdaHS), an adaptive incremental classifier, and its kernelized version: Nys-AdaHS. The classifier incorporates competitive training with a border zone. With adaptive hidden layer and tunable radii of hyper-spheres, AdaHS has strong capability of local learning like instance-based algorithms, but free from slow searching speed and excessive memory consumption. The experiments showed that AdaHS is robust, adaptive, and highly accurate. It is especially suitable for dynamic data in which patterns are changing, decision borders are complicated, and instances with the same label can be spherically clustered. [ABSTRACT FROM AUTHOR]

In this era of big data, stream data classification which is one of typical data stream applications has become more and more significant and challengeable. In these applications, it is obvious that data classification is much more frequent than model training. The ratio of stream data to be classified is rapid and time-varying, so it is an important problem to classify the stream data efficiently with high throughput. In this paper, we first analyze and categorize the current data stream machine learning algorithms according to their data structures. Then, we propose stream data classification topology (SDC-Topology) on Storm. For the classification algorithms based on the matrix, we propose self-adaptive stream data classification framework (SASDC-Framework) for efficient stream data classification on Storm. In SASDC-Framework, all the data sets arriving at the same unit time are partitioned into subsets with the nearly best partition size and processed in parallel. To select the nearly best partition size for the stream data sets efficiently, we adopt bisection method strategy and inverse distance weighted strategy. Extreme learning machine, which is a fast and accurate machine learning method based on matrix calculating, is used to test the efficiency of our proposals. According to evaluation results, the throughputs based on SASDC-Framework are 8–35 times higher than those based on SDC-Topology and the best throughput is more than 40000 prediction requests per second in our environment. [ABSTRACT FROM AUTHOR]

Incremental learning is one of the most effective methods of learning accumulated data and large-scale data. The newly increased samples of the previously known works on incremental learning are usually independent and identically distributed. To study how dependent sampling methods influence the learning ability of incremental support vector machines (ISVM) algorithm, in this paper we introduce an ISVM based on Markov resampling (MR-ISVM), and give the experimental research on the learning ability of the MR-ISVM algorithm. The experimental results indicate that the MR-ISVM algorithm has not only smaller misclassification rates and sparser of the obtained classifiers, but also less total time of sampling and training compared to ISVM based on randomly independent sampling. We also compare it with other ISVM algorithms. [ABSTRACT FROM AUTHOR]

This paper presents a deep learning-based model reduction method for distributed parameter systems (DPSs). The proposed method includes three phases. In phase I, numerical or experimental data of the spatiotemporal distribution is reduced into low-dimensional representations using the deep auto-encoder (DAE). In phase II, the low-dimensional representations are used to establish the reduced-order model. In phase III, the reduced model is then used to reconstruct the high-dimensional DPS. Experimental studies are conducted to validate the proposed method. The proposed method is compared with the classical proper orthogonal decomposition method and demonstrates better modeling accuracy and efficiency in the experiments. [ABSTRACT FROM PUBLISHER]

For most sparse coding methods, data samples are first encoded as hand-crafted features, followed by another separate learning step that generates dictionary and sparse codes. However, such feature representations may not be optimally compatible with the learning process, thus producing suboptimal results. In this paper, we propose a new architecture for nonlinear dictionary learning with sparse coding, in which samples are mapped into sparse codes via carefully designed stacked auto-encoder (SAE) networks. We jointly learn a low-dimensional embedding of the data samples by means of an SAE and a dictionary in the low-dimensional space. Further, to leverage the prior knowledge, we develop a kernel regularized nonlinear dictionary learning method, which effectively incorporates the knowledge provided by the hand-crafted kernel. An iterative algorithm is developed to jointly search the solutions of the associated optimization problem and extensive experimental validations are performed to show that the proposed kernel regularized dictionary learning method achieves satisfactory performance. [ABSTRACT FROM AUTHOR]

Deep computation models achieve super performance for big data feature learning. However, training a deep computation model poses a significant challenge since a deep computation model typically involves a large number of parameters. Specially, it needs a high-performance computing server with a large-scale memory and a powerful computing unit to train a deep computation model, making it difficult to increase the size of a deep computation model further for big data feature learning on low-end devices such as conventional desktops and portable CPUs. In this paper, we propose an improved deep computation model based on the canonical polyadic decomposition scheme to compress the parameters and to improve the training efficiency. Furthermore, we devise a learning algorithm based on the back-propagation strategy to train the parameters of the proposed model. The learning algorithm can be directly performed on the compressed parameters to improve the training efficiency. Finally, we carry on the experiments on three representative datasets, i.e., CUAVE, SNAE2, and STL-10, to evaluate the performance of the proposed model by comparing with the conventional deep computation model and other two improved deep computation models based on the Tucker decomposition and the tensor-train network. Results demonstrate that the proposed model can compress parameters greatly and improve the training efficiency significantly with a low classification accuracy drop. [ABSTRACT FROM AUTHOR]

Mining massive and high-speed data streams among the main contemporary challenges in machine learning. This calls for methods displaying a high computational efficacy, with ability to continuously update their structure and handle ever-arriving big number of instances. In this paper, we present a new incremental and distributed classifier based on the popular nearest neighbor algorithm, adapted to such a demanding scenario. This method, implemented in Apache Spark, includes a distributed metric-space ordering to perform faster searches. Additionally, we propose an efficient incremental instance selection method for massive data streams that continuously update and remove outdated examples from the case-base. This alleviates the high computational requirements of the original classifier, thus making it suitable for the considered problem. Experimental study conducted on a set of real-life massive data streams proves the usefulness of the proposed solution and shows that we are able to provide the first efficient nearest neighbor solution for high-speed big and streaming data. [ABSTRACT FROM AUTHOR]

The extreme learning machine (ELM) has become one of the most important and popular algorithms of machine learning, because of its extremely fast training speed, good generalization, and universal approximation/classification capability. The proposal of hierarchical ELM (H-ELM) extends ELM from single hidden layer feedforward networks to multilayer perceptron, greatly strengthening the applicability of ELM. Generally speaking, during training H-ELM, large-scale datasets (DSTs) are needed. Therefore, how to make use of H-ELM framework in processing big data is worth further exploration. This paper proposes a parallel H-ELM algorithm based on Flink, which is one of the in-memory cluster computing platforms, and graphics processing units (GPUs). Several optimizations are adopted to improve the performance, such as cache-based scheme, reasonable partitioning strategy, memory mapping scheme for mapping specific Java virtual machine objects to buffers. Most importantly, our proposed framework for utilizing GPUs to accelerate Flink for big data is general. This framework can be utilized to accelerate many other variants of ELM and other machine learning algorithms. To the best of our knowledge, it is the first kind of library, which combines in-memory cluster computing with GPUs to parallelize H-ELM. The experimental results have demonstrated that our proposed GPU-accelerated parallel H-ELM named as GPH-ELM can efficiently process large-scale DSTs with good performance of speedup and scalability, leveraging the computing power of both CPUs and GPUs in the cluster. [ABSTRACT FROM AUTHOR]

Semisupervised scheme has emerged as a popular strategy in the machine learning community due to the expensiveness of getting enough labeled data. In this paper, a semisupervised incremental support vector machine (SE-INC-SVM) algorithm based on neighborhood kernel estimation is proposed. First, kernel regression is constructed to estimate the unlabeled data from the labeled neighbors and its estimation accuracy is discussed from the analogy with tradition RBF neural network. The incremental scheme is derived to improve the learning efficiency and reduce the computing time. Simulations for manual data set and industrial benchmark-penicillin fermentation process demonstrate the effectiveness of the proposed SE-INC-SVM method. [ABSTRACT FROM AUTHOR]

Extreme learning machines (ELMs), as “generalized” single hidden layer feedforward networks, have been proved to be effective and efficient for classification and regression problems. Traditional ELMs assume that the training and testing data are drawn from the same distribution, which however is often violated in real-world applications. In this paper, we propose a unified cross-domain ELM (CDELM) framework to address domain adaptation problems, in which the distributions of training data (source domain) and testing data (target domain) are different but related. CDELM not only fully leverages labeled source data and unlabeled target data simultaneously to construct an adaptive target classifier but also maintains the computational efficiency of ELMs. Specifically, CDELM adapts the source classifier to target domain by matching the projected means of both domains, and explores the structure property of target domain by using manifold regularization to make the final classifier more adaptable to target data. Based on the framework, two algorithms CDELM-M and CDELM-C are proposed, which aim at minimizing the marginal and conditional distribution distance between source and target domains, respectively. Moreover, CDELM-C can further enhance the classification accuracy by multiple iterations. Comprehensive experimental studies on artificial datasets and public text and image datasets demonstrate that both CDELM-M and CDELM-C are competitive with several state-of-the-art domain adaptation learning methods in terms of the classification accuracy and efficiency. [ABSTRACT FROM AUTHOR]

This paper introduces a smart assistant for professional volleyball training based on machine-learning techniques (SAETA). SAETA addresses two main aspects of elite sports coaching: 1) technical-tactical effort control, which aims at controlling exercise effort and fatigue levels and 2) exercise quality training, which complements the former by analyzing the execution of player movements. SAETA relies on a sensing infrastructure that monitors both players and their environment, and produces real-time data that is analyzed by different modules of a decision engine. Technical-tactical effort control is based on a dynamic programming model, which selects the best activity and rest durations in interval training, with the goal of maximizing effort while preventing fatigue. Exercise quality control consists of two stages. In the first stage, movements are detected by means of a $k$ -nearest neighbors classifier and in the second stage, movement intensity is classified according to recent statistical data from the player being analyzed. These analyses are reported to coaches and players in real-time. SAETA has been developed in close collaboration with the Universidad Católica San Antonio de Murcia volleyball team, which competes in the Spanish women’s premier league. Data gathered during training sessions has provided a knowledge base for the algorithms developed, and has been used for the validation of results. [ABSTRACT FROM AUTHOR]

In lifelong machine learning (LML) systems, consecutive new tasks from changing circumstances are learned and added to the system. However, sufficiently labeled data are indispensable for extracting intertask relationships before transferring knowledge in classical supervised LML systems. Inadequate labels may deteriorate the performance due to the poor initial approximation. In order to extend the typical LML system, we propose a novel hierarchical lifelong learning algorithm (HLLA) consisting of two following layers: 1) the knowledge layer consisted of shared representations and integrated knowledge basis at the bottom and 2) parameterized hypothesis functions with features at the top. Unlabeled data is leveraged in HLLA for pretraining of the shared representations. We also have considered a selective inherited updating method to deal with intertask distribution shifting. Experiments show that our HLLA method outperforms many other recent LML algorithms, especially when dealing with higher dimensional, lower correlation, and fewer labeled data problems. [ABSTRACT FROM AUTHOR]

The Internet provides the opportunity for investors to post online opinions that they share with fellow investors. Sentiment analysis of online opinion posts can facilitate both investors' investment decision making and stock companies' risk perception. This paper develops a novel sentiment ontology to conduct context-sensitive sentiment analysis of online opinion posts in stock markets. The methodology integrates popular sentiment analysis into machine learning approaches based on support vector machine and generalized autoregressive conditional heteroskedasticity modeling. A typical financial website called Sina Finance has been selected as an experimental platform where a corpus of financial review data was collected. Empirical results suggest solid correlations between stock price volatility trends and stock forum sentiment. Computational results show that the statistical machine learning approach has a higher classification accuracy than that of the semantic approach. Results also imply that investor sentiment has a particularly strong effect for value stocks relative to growth stocks. [ABSTRACT FROM AUTHOR]

This article deals with the development of four modified radial basis function neural network (RBFNN) models. The corresponding learning algorithms associated with the updating of internal parameters of the models are derived. The conventional inputs are used in the first and second modified RBFNN models (models 3 and 4) whereas exponential nonlinear inputs are used in the fifth and sixth RBFNN models to provide additional nonlinearity for achieving a better solution of nonlinear classification, and direct and inverse modeling problems. To assess and compare the performance potentiality of the proposed four new RBFNN models, one classification problem, one direct modeling problem, and one inverse modeling problem are solved through computer simulation-based experiments. For comparison and to assign the performance rank of each of the four modified RBFNN models, two conventional and commonly used RBFNN models (models 1 and 2) are also simulated. To access the performance of different models during the training phase of Examples 1 and 2, the root mean-square error (RMSE) value, mean absolute deviation (MAD), and the number of iterations required to achieve convergence are obtained. For the third example, only the first two performance measures are found. During the testing or validation phase, the output responses of the different models of Example 2 are compared with the desired response analysis. For Example 3, the bit-error rate (BER) plots are compared. The observation of all the results demonstrates consistent ranks of all models in the case of all three examples. It is, in general, found that the ranks of the models 1–6 are 6, 4, 3, 2, 5, and 1, respectively. In essence, in terms of all performance measures, model M-6 with an exponential version of inputs with weights on both layers occupies the first position whereas model M-4 with conventional inputs as the second position. [ABSTRACT FROM AUTHOR]

Complex-valued neural network is a kind of learning model which can deal with problems in complex domain. Fully complex extreme learning machine (CELM) is a much faster training algorithm than the complex backpropagation (CBP) scheme. However, it is at the cost of using more hidden nodes to obtain the comparable performance. An upper-layer-solution-aware algorithm has been proposed for training single-hidden layer feedforward neural networks, which performs much better than its counterparts, pseudo-inverse learning (PIL)/extreme learning machine and gradient decent-based backpropagation neural networks. Consequently, there exist two challenges that need to be dealt with: 1) How to combine the advantages of CBP and CELM to develop a novel complex learning algorithm? and 2) What is the convergent behavior of the presented algorithm? In this article, an input weights dependent complex-valued (IWDCV) learning algorithm based on Wirtinger calculus has been proposed, which effectively solves the nonanalytic problem of the common activation functions during training neural networks. In addition, the monotonicity of the error function and the deterministic convergence of the proposed model have been strictly proved, which theoretically guarantee the efficiency and effectiveness of the given model, IWDCV. Finally, for real and complex-valued problems, a variety of simulations have been done to demonstrate the comparable performance of the proposed algorithm which support the theoretical observations as well. [ABSTRACT FROM AUTHOR]

This article investigates hardware implementation of hierarchical temporal memory (HTM), a brain-inspired machine learning algorithm that mimics the key functions of the neocortex and is applicable to many machine learning tasks. Spatial pooler (SP) is one of the main parts of HTM, designed to learn the spatial information and obtain the sparse distributed representations (SDRs) of input patterns. The other part is temporal memory (TM) which aims to learn the temporal information of inputs. The memristor, which is an appropriate synapse emulator for neuromorphic systems, can be used as the synapse in SP and TM circuits. In this article, a memristor-based SP (MSP) circuit structure is designed to accelerate the execution of the SP algorithm. The presented MSP has properties of modeling both the synaptic permanence and the synaptic connection state within a single synapse, and on-device and parallel learning. Simulation results of statistic metrics and classification tasks on several real-world datasets substantiate the validity of MSP. [ABSTRACT FROM AUTHOR]

A fast architecture for real-time (i.e., minute-based) training of a traffic predictor is studied, based on the so-called broad learning system (BLS) paradigm. The study uses various traffic datasets by the California Department of Transportation, and employs a variety of standard algorithms (LASSO regression, shallow and deep neural networks, stacked autoencoders, convolutional, and recurrent neural networks) for comparison purposes: all algorithms are implemented in MATLAB on the same computing platform. The study demonstrates a BLS training process two-three orders of magnitude faster (tens of seconds against tens-hundreds of thousands of seconds), allowing unprecedented real-time capabilities. Additional comparisons with the extreme learning machine architecture, a learning algorithm sharing some features with BLS, confirm the fast training of least-square training as compared to gradient training. [ABSTRACT FROM AUTHOR]

Extreme learning machines (ELMs) has been theoretically and experimentally proved to achieve promising performance at a fast learning speed for supervised classification tasks. However, it does not perform well on imbalanced binary classification tasks and tends to get biased toward the majority class. Besides, since a large amount of training data with labels are not always available in the real world, there is an urgent demand to develop an efficient semi-supervised version of ELM for imbalanced binary classification tasks. In this article, owing to the distinct insensitivity of area under the ROC curve (AUC) to both class skews and changes of class distributions, we focus the study on integrating AUC maximization into the ELM framework to tackle with imbalanced binary classification tasks well. By demystifying the AUC metric with the ELM framework, we develop a new AUC-based ELM called AUC-ELM for imbalanced binary classification, which essentially is revealed to be equivalent to an ELM on another transformed data space. Accordingly, its semi-supervised version called SAUC-ELM is also developed. Both AUC-ELM and SAUC-ELM have the distinctive merits: 1) they share the advantage of ELM in both generalization capability and training efficiency, and further uniquely tailored for imbalanced binary classification tasks and 2) in contrast to the existing imbalanced variants of ELM, such as class-specific cost regulation ELM and semi-supervised ELM, they have fewer parameters to tune, thereby reducing the computational cost for model selection. Experiments on a heap of datasets show that both AUC-ELM and SAUC-ELM outperform the other comparative methods in terms of both classification performance and training speed. [ABSTRACT FROM AUTHOR]

Once defined, the access control policies and regulations would never be changed in a running and state transition process. However, it will give attackers the possibility of discovering vulnerabilities in the system, and the control systems lack the ability of dynamic perception of security state and risk, causing the systems to be exposed to risks. In this article, a dynamic Bell–LaPadula (BLP) model is proposed. The conditional random field (CRF) is introduced into the BLP model to optimize the rules. First, the model formalizes the security attributes, states of system, transition rules, and constraint models on the basis of the state transition of CRFs. After the historical system access logs are processed as the original dataset, a feature selection method is proposed to extract the requests and current states as feature vectors. Second, this article presents a rules training algorithm based on L-BFGS to implement the study and training of datasets, and then marks the logs in the test set through Viterbi algorithm automatically. On the base of these, a rule generation algorithm is proposed to dynamically adjust the access control rules based on the current security status and events of the system. Third, the security of CRFs-BLP is proved by theoretical analysis. Finally, the validity and accuracy of the model are verified by estimating the value of the precision, recall, and $F1$ -score. As the system threats are shown to be decreased obviously from these experiments, this dynamic model can decrease the vulnerabilities and risk effectively. [ABSTRACT FROM AUTHOR]

Fault diagnosis of rolling bearings is an essential process for improving the reliability and safety of the rotating machinery. It is always a major challenge to ensure fault diagnosis accuracy in particular under severe working conditions. In this article, a deep adversarial domain adaptation (DADA) model is proposed for rolling bearing fault diagnosis. This model constructs an adversarial adaptation network to solve the commonly encountered problem in numerous real applications: the source domain and the target domain are inconsistent in their distribution. First, a deep stack autoencoder (DSAE) is combined with representative feature learning for dimensionality reduction, and such a combination provides an unsupervised learning method to effectively acquire fault features. Meanwhile, domain adaptation and recognition classification are implemented using a Softmax classifier to augment classification accuracy. Second, the effects of the number of hidden layers in the stack autoencoder network, the number of neurons in each hidden layer, and the hyperparameters of the proposed fault diagnosis algorithm are analyzed. Third, comprehensive analysis is performed on real data to validate the performance of the proposed method; the experimental results demonstrate that the new method outperforms the existing machine learning and deep learning methods, in terms of classification accuracy and generalization ability. [ABSTRACT FROM AUTHOR]

The ever-increasing complexity of robot applications induces the need for methods to approach problems with no (viable) analytical solution. Deep learning (DL) provides a set of tools to address this kind of problems. This survey presents a categorization of the major challenges in robotics that leverage DL technologies and introduces representative examples of successful solutions for the described problems. We also consider the question when and whether to use modular, monolithic models or end-to-end DL, in order to provide a guideline for the selection of the correct model structure and training strategy. By doing so, the current role and adaptability of different techniques at different hierarchical levels of a robot-application can be highlighted, thus providing a well-structured basis to assist future approaches. [ABSTRACT FROM AUTHOR]