Search

Sequential decision-making tasks that require information integration over extended durations of time are challenging for several reasons, including the problem of vanishing gradients, long training times and significant memory requirements. To this end, we propose a neuron model fashioned after the JK flip-flops in digital systems. A flip-flop is a sequential device that can store state information of the previous history. We incorporate the JK flip-flop neuron into several deep network architectures and apply the networks to difficult sequence processing problems. The proposed architectures include flip-flop neural networks (FFNNs), bidirectional flip-flop neural networks (BiFFNNs), convolutional flip-flop neural networks (ConvFFNNs), and bidirectional convolutional flip-flop neural networks (BiConvFFNNs). Learning rules of proposed architectures have also been derived. We have considered the most popular benchmark sequential tasks like signal generation, sentiment analysis, handwriting generation, text generation, video frame prediction, lung volume prediction, and action recognition to evaluate the proposed networks. Finally, we compare the results of our networks with the results from analogous networks with Long Short-Term Memory (LSTM) neurons on the same sequential tasks. Our results show that the JK flip-flop networks outperform the LSTM networks significantly or marginally on all the tasks, with only half of the trainable parameters. [ABSTRACT FROM AUTHOR]

A direct adaptive neural control scheme for a class of nonlinear systems is presented in the paper. The proposed control scheme incorporates a neural controller and a sliding mode controller. The neural controller is constructed based on the approximation capability of the single-hidden layer feedforward network (SLFN). The sliding mode controller is built to compensate for the modeling error of SLFN and system uncertainties. In the designed neural controller, its hidden node parameters are modified using the recently proposed neural algorithm named extreme learning machine (ELM), where they are assigned random values. However, different from the original ELM algorithm, the output weight is updated based on the Lyapunov synthesis approach to guarantee the stability of the overall control system. The proposed adaptive neural controller is finally applied to control the inverted pendulum system with two different reference trajectories. The simulation results demonstrate good tracking performance of the proposed control scheme. [ABSTRACT FROM AUTHOR]

The extreme learning machine (ELM) is a method to train single-layer feed-forward neural networks that became popular because it uses a fast closed-form expression for training that minimizes the training error with good generalization ability to new data. The ELM requires the tuning of the hidden layer size and the calculation of the pseudo-inverse of the hidden layer activation matrix for the whole training set. With large-scale classification problems, the computational overload caused by tuning becomes not affordable, and the activation matrix is extremely large, so the pseudo-inversion is very slow and eventually the matrix will not fit in memory. The quick extreme learning machine (QELM), proposed in the current paper, is able to manage large classification datasets because it: (1) avoids the tuning by using a bounded estimation of the hidden layer size from the data population; and (2) replaces the training patterns in the activation matrix by a reduced set of prototypes in order to avoid the storage and pseudo-inversion of large matrices. While ELM or even the linear SVM cannot be applied to large datasets, QELM can be executed on datasets up to 31 million data, 30,000 inputs and 131 classes, spending reasonable times (less than 1 h) in general purpose computers without special software nor hardware requirements and achieving performances similar to ELM. [ABSTRACT FROM AUTHOR]

The impulsive complex-valued neural networks (CVNNs) with multi-proportional delays (MPDs) are considered. By setting up a suitable Lyapunov–Krasovskii functional (L-KF) and utilizing the matrix inequality skills, several delay-dependent criteria for examining the global polynomial stability (GPS) of the CVNNs are built via linear matrix inequalities (LMIs), which can be verified numerically using the valid YALMIP toolbox in MATLAB. An example with simulations is presented to highlight the potency and the efficiency of the raised criteria. [ABSTRACT FROM AUTHOR]

Artificial neural network training algorithms aim to optimize the network parameters regarding the pre-defined cost function. Gradient-based artificial neural network training algorithms support iterative learning and have gained immense popularity for training different artificial neural networks end-to-end. However, training through gradient methods is time-consuming. Another family of training algorithms is based on the Moore–Penrose inverse, which is much faster than many other gradient methods. Nevertheless, most of those algorithms are non-iterative and thus do not support mini-batch learning in nature. This work extends two non-iterative Moore–Penrose inverse-based training algorithms to enable online sequential learning: a single-hidden-layer autoencoder training algorithm and a sub-network-based classifier training algorithm. We further present an approach that uses the proposed autoencoder for self-supervised dimension reduction and then uses the proposed classifier for supervised classification. The experimental results show that the proposed approach achieves satisfactory classification accuracy on many benchmark datasets with extremely low time consumption (up to 50 times faster than the support vector machine on CIFAR 10 dataset). [ABSTRACT FROM AUTHOR]

Our research is devoted to answering whether randomisation-based learning can be fully competitive with the classical feedforward neural networks trained using backpropagation algorithm for classification and regression tasks. We chose extreme learning as an example of randomisation-based networks. The models were evaluated in reference to training time and achieved efficiency. We conducted an extensive comparison of these two methods for various tasks in two scenarios: ∙ using comparable network capacity and ∙ using network architectures tuned for each model. The comparison was conducted on multiple datasets from public repositories and some artificial datasets created for this research. Overall, the experiments covered more than 50 datasets. Suitable statistical tests supported the results. They confirm that for relatively small datasets, extreme learning machines (ELM) are better than networks trained by the backpropagation algorithm. But for demanding image datasets, like ImageNet, ELM is not competitive to modern networks trained by backpropagation; therefore, in order to properly address current practical needs in pattern recognition entirely, ELM needs further development. Based on our experience, we postulate to develop smart algorithms for the inverse matrix calculation, so that determining weights for challenging datasets becomes feasible and memory efficient. There is a need to create specific mechanisms to avoid keeping the whole dataset in memory to compute weights. These are the most problematic elements in ELM processing, establishing the main obstacle in the widespread ELM application. [ABSTRACT FROM AUTHOR]

The real-time availability of key water quality parameters is of great importance for an advanced and optimized process control in wastewater treatment plants (WWTPs). However, due to the complex environment conditions and costly measuring instruments, it is generally difficult and time-consuming to measure certain key water quality parameters online, such as the effluent biochemical oxygen demand (BOD) and the effluent total nitrogen (TN). Recently, artificial neural networks have powered the online prediction tasks in several WWTPs. Hence, in this paper, an adaptive task-oriented radial basis function (ATO-RBF) network is developed to design prediction models for accurate timely acquirements of the effluent BOD and the effluent TN. The advantage of ATO-RBF network is that the architecture is not designed by human engineers; it is adaptively generated from the data to be processed. First, to enhance the learning ability and generalization performance of prediction models, an error correction-based growing strategy and a second-order learning algorithm are combined to design the ATO-RBF network. Then, RFB nodes with low significance would be pruned without sacrificing the learning accuracy, making the prediction model more compact. Additionally, the convergence of the ATO-RBF network is analyzed based on the Lyapunov criterion, which can guarantee its feasibility in practical applications. Finally, the proposed methodology is verified by benchmark simulations and real industrial data, showing superior prediction accuracy in compared with conventional methods. [ABSTRACT FROM AUTHOR]

The transmission of weather information of a location at certain time intervals affects the living conditions of the people there directly or indirectly. According to weather information, people shape their behavior in daily life. Besides, agricultural activities are carried out according to the weather conditions. Considering the importance of this subject, it is possible to make weather predictions based on the weather images in today's technology exploiting the computer systems. However, the recent mention of the name of artificial intelligence technology in every field has made it compulsory for computer systems to benefit from this technology. The dataset used in the study has four classes: cloudy, rain, shine, and sunrise. In the study, GoogLeNet and VGG-16 models and the spiking neural network (SNN) were used together. The features extracted from GoogLeNet and VGG-16 models were combined and given to the SNNs as the input. As a result, the SNNs contributed to the success of classification with the proposed approach. The classification accuracy rates of cloudy, rain, shine, and sunrise classes were 98.48%, 97.58%, 97%, and 98.48%, respectively, together with SNN. Also, the use of SNNs in combination with deep learning models to obtain a successful result is proved in this study. [ABSTRACT FROM AUTHOR]

MRI is a broadly used imaging method to determine glioma-based tumors. During image processing, MRI provides large image information, and therefore, an accurate image processing must be carried out in clinical practices. Therefore, automatic and consistent methods are requisite for knowing the precise details of the image. The automated segmentation method inheres obstacles like inconsistency in tracing out the large spatial and structural inconsistency of brain tumors. In this work, a semantic-based U-NET-convolutional neural networks exploring a 3∗3 kernel's size is proposed. Small kernels have an effect against overfitting in the deeper architecture and provide only a smaller number of weights in this network. Multiscale multimodal convolutional neural network (MSMCNN) with long short-term memory (LSTM)-based deep learning semantic segmentation technique is used for multimodalities of magnetic resonance images (MRI). The proposed methodology aims to identify and segregate the classes of tumors by analyzing every pixel in the image. Further, the performance of semantic segmentation is enhanced by applying a patch-wise classification technique. In this work, multiscale U-NET-based deep convolution network is used for classifying the multimodal convolutions into three different scale patches based on a pixel level. In order to identify the tumor classes, all three pathways are combined in the LSTM network. The proposed methodology is validated by a fivefold cross-validation scheme from MRI BRATS'15 dataset. The experiment outcomes show that the MSMCNN model outperforms the CNN-based models over the Dice coefficient and positive predictive value and obtains 0.9214 sensitivity and 0.9636 accuracy. [ABSTRACT FROM AUTHOR]

During the last decade, a significant research progress has been drawn in both the theoretical aspects and the applications of Deep Learning Neural Networks. Besides their spectacular applications, optimal architectures of these neural networks may speed up the learning process and exhibit better generalization results. So far, many growing and pruning algorithms have been proposed by many researchers to deal with the optimization of standard Feedforward Neural Network architectures. However, applying both the growing and the pruning on the same net may lead a good model for a big data set and hence good selection results. This work is devoted to propose a new Growing and pruning Learning algorithm for Deep Neural Networks. This new algorithm is presented and applied on diverse medical data sets. It is shown that this algorithm outperforms various other artificial intelligent techniques in terms of accuracy and simplicity of the resulting architecture. [ABSTRACT FROM AUTHOR]

Feedforward neural network (FNN) is the most popular network model, and the appropriate structure and learning algorithms are the key of its performance. This paper proposes an online self-organizing algorithm for feedforward neural network (OSNN) with a single hidden layer. The proposed OSNN optimizes the structure of FNN for time-varying system including structure design and parameter learning. In structure design, this paper measures the contribution ratios of hidden nodes by local sensitivity analysis based on differentiation method. OSNN merges hidden nodes with the others that have the highest correlation when their contribution ratios are almost zero and adds new hidden nodes by error reparation. For parameter learning, an improved online gradient method (OGM), called online gradient method with fixed memory (FMOGM), is proposed to improve the convergence speed and accuracy of OGM. In addition, this paper calculates the contribution ratios and the network error and estimates the local minima by using the fixed-sized training set of FMOGM instead of one sample at the current time, which can obtain more effective local information and a compact network structure. Finally, the proposed OSNN is verified using a number of benchmark problems and a practical problem for biochemical oxygen demand prediction in wastewater treatment. The experimental results show that OSNN has better convergence speed and accuracy than other algorithms. [ABSTRACT FROM AUTHOR]

Hands are the main environmental manipulator for the human being. After losing a hand, the only alternative for the victim is to use a prosthesis. Despite the progress of science, the modern prosthesis has the same age-old problem of accurate force estimation. Among different kinds of force, clench force is the most important one. Because of this importance, this paper presents a hardware system that has been designed and implemented to estimate the desired clench force using surface Electromyography signals recorded from lower-arm muscles. The implementation includes a two-layer artificial neural network with a surface electromyography integrator. The neural network was trained with the Levenberg–Marquardt back propagation algorithm and was implemented in a field programmable gate array using an off-chip training method. The results from 10 datasets, recorded from five subjects, show that the hardware model is very accurate, with an average mean square error of 0.003. This suggests that the proposed design can mimic the behavior of clench force that a real limb does, and therefore this intelligent system could be a useful tool for any application related to prostheses. [ABSTRACT FROM AUTHOR]

Object affordance detection, which aims to understand functional attributes of objects, is of great significance for an autonomous robot to achieve a humanoid object manipulation. In this paper, we propose a novel relationship-aware convolutional neural network, which takes the symbiotic relationship between multiple affordances and the combinational relationship between the affordance and objectness into consideration, to predict the most probable affordance label for each pixel in the object. Different from the existing CNN-based methods that rely on separate and intermediate object detection step, our proposed network directly produces the pixel-wise affordance maps from an input image in an end-to-end manner. Specifically, there are three key components in our proposed network: Coord-ASPP module introducing CoordConv in atrous spatial pyramid pooling (ASPP) to refine the feature maps, relationship-aware module linking the affordances and corresponding objects to explore the relationships, and online sequential extreme learning machine auxiliary attention module focusing on individual affordances further to assist relationship-aware module. The experimental results on two public datasets have shown the merits of each module and demonstrated the superiority of our relationship-aware network against the state of the arts. [ABSTRACT FROM AUTHOR]

As an important learning algorithm, extreme learning machine (ELM) is known for its excellent learning speed. With the expansion of ELM's applications in the field of classification and regression, the need for its real-time performance is increasing. Although the use of hardware acceleration is an obvious solution, how to select the appropriate acceleration hardware for ELM-based applications is a topic worthy of further discussion. For this purpose, we designed and evaluated the optimized ELM algorithms on three kinds of state-of-the-art acceleration hardware, i.e., multi-core CPU, Graphics Processing Unit (GPU), and Field-Programmable Gate Array (FPGA) which are all suitable for matrix multiplication optimization. The experimental results showed that the speedup ratio of these optimized algorithms on acceleration hardware achieved 10–800. Therefore, we suggest that (1) use GPU to accelerate ELM algorithms for large dataset, and (2) use FPGA for small dataset because of its lower power, especially for some embedded applications. We also opened our source code. [ABSTRACT FROM AUTHOR]

This paper aims to improve data-driven prognostics by presenting a novel approach of directly estimating the remaining useful life (RUL) of aero-engines without requiring setting any failure threshold information or estimating degradation states. Specifically, based on the sensory data, RUL estimations are directly obtained through the universal function approximation capability of the extreme learning machine (ELM) algorithm. To achieve this, the features related with the RUL are first extracted from the sensory data as the inputs of the ELM model. Besides, to optimize the number of observed sensors, three evaluation metrics of correlation, monotonicity and robustness are defined and combined to automatically select the most relevant sensor values for more effective and efficient remaining useful life predictions. The validity and superiority of the proposed approach is evaluated by the widely used turbofan engine datasets from NASA Ames prognostics data repository. The proposed approach shows improved RUL estimation applicability at any time instant of the degradation process without determining the failure thresholds. This also simplifies the RUL estimation procedure. Moreover, the random properties of hidden nodes in the ELM learning mechanisms ensures the simplification and efficiency for real-time implementation. Therefore, the proposed approach suits to real-world applications in which prognostics estimations are required to be fast. [ABSTRACT FROM AUTHOR]

Mu rhythm is a spontaneous neural response occurring during a motor imagery (MI) task and has been increasingly applied to the design of brain–computer interface (BCI). Accurate classification of MI is usually rather difficult to be achieved since mu rhythm is very weak and likely to be contaminated by other background noises. As an extension of the single layer feedforward network, extreme learning machine (ELM) has recently proven to be more efficient than support vector machine that is a benchmark for MI-related EEG classification. With probabilistic inference, this study introduces a sparse Bayesian ELM (SBELM)-based algorithm to improve the classification performance of MI. SBELM is able to automatically control the model complexity and exclude redundant hidden neurons by combining advantageous of both ELM and sparse Bayesian learning. The effectiveness of SBELM for MI-related EEG classification is validated on a public dataset from BCI Competition IV IIb in comparison with several other competing algorithms. Superior classification accuracy confirms that the proposed SBELM-based algorithm is a promising candidate for performance improvement of an MI BCI. [ABSTRACT FROM AUTHOR]

Today's industry demands precise functioning and zero failure of rotating machinery (RM) to avoid disastrous accidents as well as financial losses. Rolling element bearings (REBs) are the heart of RM. Therefore, as early as possible to provide the significant time for maintenance planning, an intelligent diagnosis of REB fault is a critical and challenging task. Thus, this paper presents an efficient method for fault diagnosis. The proposed method mainly consists of two consecutive units: (1) generation of kurtogram of raw vibration signal and (2) training of extreme learning machine (ELM) classifier using kurtogram. Kurtogram has a distinct capability to represent the hidden non-stationary components of a raw signal. Therefore, it is considered as a unique feature vector for fault classification. ELM is a well-organized fast learning method proposed by Huang et al. and showed that it is better than traditional learning algorithms. However, one of the open issues of ELM is to design compact-size ELM architecture by preserving the accuracy of the solution. Thus, improved random increment ELM is proposed in this paper. Initially, it randomly adds the nodes to network architecture to rapidly reduce the residual error up to the predefined threshold and then sequentially adds the nodes to the network architecture for further reducing the residual error. Performance of the proposed routine is evaluated by REB vibration data: artificially generated vibration data and Case Western Reserve University bearing data. The experimental study reveals the classification accuracy of the proposed approach with both the datasets for various faults and also compared with existing methods. [ABSTRACT FROM AUTHOR]

Based on a novel algorithm, known as the upper-layer-solution-aware (USA), a new algorithm, in which the penalty method is introduced into the empirical risk, is studied for training feed-forward neural networks in this paper, named as USA with penalty. Both theoretical analysis and numerical results show that it can control the magnitude of weights of the networks. Moreover, the deterministic theoretical analysis of the new algorithm is proved. The monotonicity of the empirical risk with penalty term is guaranteed in the training procedure. The weak and strong convergence results indicate that the gradient of the total error function with respect to weights tends to zero, and the weight sequence goes to a fixed point when the iterations approach positive infinity. Numerical experiment has been implemented and effectively verifies the proved theoretical results. [ABSTRACT FROM AUTHOR]

Short-term electricity price forecasting in deregulated electricity markets has been studied extensively in recent years but without significant reduction in price forecasting errors. Also demand-side management and short-term scheduling operations in smart grids do not require strictly very accurate forecast and can be executed with certain practical price thresholds. This paper, therefore, presents a multikernel extreme learning machine (MKELM) for both short-term electricity price forecasting and classification according to some prespecified price thresholds. The kernel ELM does not require the hidden layer mapping function to be known and produces robust prediction and classification in comparison with the conventional ELM using random weights between the input and hidden layers. Further in the MKELM formulation, the linear combination of the weighted kernels is optimized using vaporization precipitation-based water cycle algorithm (WCA) to produce significantly accurate electricity price prediction and classification. The combination of MKELM and WCA is named as WCA-MKELM in this work. To validate the effectiveness of the proposed approach, three electricity markets, namely PJM, Ontario and New South Wales, are considered for electricity price forecasting and classification producing fairly accurate results. [ABSTRACT FROM AUTHOR]

In this study, a modular design methodology inherited from cognitive neuroscience and neurophysiology is proposed to develop artificial neural networks, aiming to realize the powerful capability of brain—divide and conquer—when tackling complex problems. First, a density-based brain-like partition method is developed to construct the modular architecture, with a highly connected center in each sub-network as the human brain. The whole task is also divided into different sub-tasks at this stage. Then, a compact radial basis function (RBF) network with fast learning speed and desirable generalization performance is applied as the sub-network to solve the corresponding task. On the one hand, the modular structure helps to improve the ability of neural networks on complex problems by implementing divide and conquer. On the other hand, sub-networks with considerable ability could guarantee the parsimonious and generalization of the entire neural network. Finally, the novel modular RBF (NM-RBF) network is evaluated through multiple benchmark numerical experiments, and results demonstrate that the NM-RBF network is capable of constructing a relative compact architecture during a short learning process with achievable satisfactory generalization performance, showing its effectiveness and outperformance. [ABSTRACT FROM AUTHOR]

Parameter settings for nature-inspired optimization algorithms are essential for their effective performance. Evolutionary algorithms and swarm intelligence algorithms are prominent types of nature-inspired optimization. There are comprehensive reviews of parameter setting techniques for evolutionary algorithms. Counterparts providing an overview of parameter setting techniques for swarm intelligence algorithms are needed also. Therefore, in this paper, we provide a critical and comprehensive review, focusing in particular on dynamic parameter setting techniques. The paper describes a variety of swarm intelligence algorithms and parameter setting approaches that have been applied to them. This review simplifies the selection of parameter setting techniques for each algorithm by collecting them in a single document and classifying them under a taxonomy. Recommendations for parameter setting approach selection are provided in this review. We explore the open problems related to dynamic parameter setting techniques for swarm intelligence optimization and discuss the trade-off between run-time computation and flexibility of these algorithms. [ABSTRACT FROM AUTHOR]

In the present scenario of technological breakthroughs in the automotive industry, machine learning is greatly contributing to the development of safer and more comfortable vehicles. In particular, personalization of the driving experience using machine learning is an innovative trend that comprises the development of both customized driver assistance systems and in-cabin comfort features. In this work, a versatile hardware/software platform for personalized driver assistance, using online sequential extreme learning machines (OS-ELM), is presented. The system, based on a programmable system-on-chip (SoC), is able to recognize the driver and personalize the behavior of the car. The platform provides high speed, small size, efficient power consumption, and true capability for real-time adaptation (i.e., on-chip self-learning). In addition, due to the plasticity and scalability of the OS-ELM algorithm and the programmable nature of the SoC, this solution is flexible enough to cope with the incremental changes that the new generation of vehicles are demanding. The implementation details of a system, suitable for current levels of driving automation, are provided. [ABSTRACT FROM AUTHOR]

How to deeply process market data sources and build systems to process accurate market impact analysis is an attractive problem. In this paper, we build up a system that exploits deep learning architecture to improve feature representations, and adopt state-of-the-art supervised learning algorithm—extreme learning machine—to predict market impacts. We empirically evaluate the performance of the system by comparing different configurations of representation learning and classification algorithms, and conduct experiments on the intraday tick-by-tick price data and corresponding commercial news archives of stocks in Hong Kong Stock Exchange. From the results, we find that in order to make system achieve good performance, both the representation learning and the classification algorithm play important roles, and comparing with various benchmark configurations of the system, deep learned feature representation together with extreme learning machine can give the highest market impact prediction accuracy. [ABSTRACT FROM AUTHOR]

The underwater thruster is considered one of the most critical components located on an unmanned underwater vehicle to maneuver in the water. However, it is recognized as a common source of the fault. This phenomenon is made worse when collected data for equipment health diagnostics are highly imbalanced. A new sampling method to tackle the problem of imbalanced data based on cosine similarity is proposed to improve the classification accuracy for thruster health diagnostics. The results show that it outperforms SMOTE (Synthetic Minority Oversampling Technique) and ADASYN (Adaptive Synthetic Sampling Approach for Imbalanced Learning). The proposed method was further validated using different imbalanced datasets with different imbalance ratio from KEEL and UCI machine learning repository (such as Pima Indians Diabetes, Ionosphere, Fertility Diagnostics, Mammographic Masses, Blood Transfusion Service Centre). The majority of the results from the datasets show that the proposed method produces the higher classification accuracy as well as g-means that suggests the potential approach for classification problem that has a highly imbalanced dataset. [ABSTRACT FROM AUTHOR]

A tensor-based extreme learning machine is proposed, which is referred to as tensor-based type-2 extreme learning machine (TT2-ELM). In contrast to the work on ELM, regularized ELM (RELM), weighted regularized ELM (WRELM) and least squares support vector machine (LS-SVM), which are the most often used learning algorithm in regression problems, TT2-ELM adopts the tensor structure to construct the ELM for type-2 fuzzy sets, Moore–Penrose inverse of tensor is used to obtain the tensor regression result. No further type-reduction method is needed to obtain the coincide type-1 fuzzy sets, and type-2 fuzzy structure can be seamlessly incorporated into the ELM scheme. Experimental results are carried out on two Sinc functions, a nonlinear system identification problem and four real-world regression problems, results show that TT2-ELM performs at competitive level of generalized performance as the ELM, RELM, WRELM and LS-SVM on the small- and moderate-scale data sets. [ABSTRACT FROM AUTHOR]

A clustering degeneracy algorithm, called SpikeCD, with spiking RBF neurons for classification is proposed in this paper. Unlike traditional spiking RBF networks where their performance severely relies on the time-costing process of parameter optimization, SpikeCD uses a clustering degeneracy strategy to adjust the number and centers of spiking RBF neurons, which is insensitive to parameters. A supervised learning is followed to improve network's classification ability. Its performance is demonstrated on several benchmark datasets from the UCI Machine Learning Repository and image datasets. The results show SpikeCD can achieve good classification accuracy with simple structure. Moreover, the variation of parameters has a little effect on it. We hope this algorithm can be a new inspiration for improving the robustness of evolving spiking neural networks and other machine learning methods. [ABSTRACT FROM AUTHOR]

In this study, two-channel surface electromyography (sEMG) signals were used to classify hand finger movements. Bicoherence analysis of the sEMG signal recorded with surface electrodes for flexor and extensor muscle bundles on the front and back of the forearm, respectively, was classified by extreme learning machines (ELM) based on phase matches in the electromyography (EMG) signal. EMG signals belonging to 42 human, 22 males and 20 females, with an average age of 21.4 were used in the study. The finger movements were also classified by using different learning algorithms. The best classification was performed by using ELM algorithm with 98.95 and 97.83% accuracies in average for subjects individually and all together, respectively. On the other hand, a maximum of 95.81 and 94.30% accuracies were reached for subjects individually and all together, respectively, with other learning methods used in the present study. From the information obtained through this study, it is possible to control finger movements by using flexor and extensor muscle activities of the forearm. Furthermore, by this method, it may be possible controlling of the intelligent prosthesis hands with high degree of freedom. [ABSTRACT FROM AUTHOR]

This paper proposes a more reasonable objective function for combined economic emission dispatch problem. To solve it, Lagrange programming neural network (LPNN) is utilized to obtain optimal scheduling of a hybrid microgrid, which includes power generation resources, variable demands and energy storage system for energy storing and supplying. Combining variable neurons with Lagrange neurons, the LPNN aims to minimize the cost function and maximize the power generated by the renewable sources. The asymptotic stability condition of the neurodynamic model is analyzed, and simulation results show that optimal power of each component with certain time interval can be obtained. In addition, a new method by radial basis function neural network is proposed to predict the power values of renewable energy and load demand, which are used as the input values in the optimal process. [ABSTRACT FROM AUTHOR]

Incremental learning is a technique which is effective to increase the space efficiency of machine learning algorithms. Ensemble learning can combine different algorithms to form more accurate ones. The parameter selection of incremental methods is difficult because no retraining is allowed, and the combination of incremental and ensemble learning has not been fully explored. In this paper, we propose a parameter-free regression framework and it combines incremental learning and ensemble learning. First, the topology learning neural networks such as growing neural gas (GNG) and self-organizing incremental neural network (SOINN) are employed as solutions to nonlinearity. Then, the vector quantizations of GNG and SOINN are transformed into a feed-forward neural network by an improved Nadaraya–Watson estimator. A maximum likelihood process is devised for adaptive parameter selection of the estimator. Finally, a weighted training strategy is incorporated to enable the topology learning regressors for ensemble learning by AdaBoost. Experiments are carried out on 5 UCI datasets, and an application study of short-term traffic flow prediction is given. The results show that the proposed method gives comparable results to mainstream incremental and non-incremental regression methods, and better performances in the short-term traffic flow prediction. [ABSTRACT FROM AUTHOR]

Electroencephalography (EEG) has been used as a promising tool for investigation of brain activity during cognitive processes. The aim of this study is to reveal whether EEG signals can be used for classifying cognitive processes: arithmetic tasks and text reading. A recently introduced EEG database, which is constructed from 18 healthy subjects during a slide show including 60 slides of simple arithmetic tasks and easily readable texts, is used for this purpose. Multi-order difference plot-based time-domain attributes, number of values in specified regions after scattering the sequential difference values with several degrees, are extracted. For classification, improved extreme learning machine (ELM) scheme, namely luELM, by the use of lower–upper triangularization method instead of singular value decomposition which has disadvantages when used with huge data is proposed. As a result, higher accuracy results are achieved with reduced training time for proposed luELM classifier than traditional ELM classifier for both subject-dependent and subject-independent analysis. [ABSTRACT FROM AUTHOR]

This paper investigates the distributed cooperative learning (DCL) problems over networks, where each node only has access to its own data generated by the unknown pattern (map or function) uniformly, and all nodes cooperatively learn the pattern by exchanging local information with their neighboring nodes. These problems cannot be solved by using traditional centralized algorithms. To solve these problems, two novel DCL algorithms using wavelet neural networks are proposed, including continuous-time DCL (CT-DCL) algorithm and discrete-time DCL (DT-DCL) algorithm. Combining the characteristics of neural networks with the properties of the wavelet approximation, the wavelet series are used to approximate the unknown pattern. The DCL algorithms are used to train the optimal weight coefficient matrix of wavelet series. Moreover, the convergence of the proposed algorithms is guaranteed by using the Lyapunov method. Compared with existing distributed optimization strategies such as distributed average consensus (DAC) and alternating direction method of multipliers (ADMM), our DT-DCL algorithm requires less information communications and training time than ADMM strategy. In addition, it achieves higher accuracy than DAC strategy when the network consists of large amounts of nodes. Moreover, the proposed CT-DCL algorithm using a proper step size is more accurate than the DT-DCL algorithm if the training time is not considered. Several illustrative examples are presented to show the efficiencies and advantages of the proposed algorithms. [ABSTRACT FROM AUTHOR]

In order to curb the model expansion of the kernel learning methods and adapt the nonlinear dynamics in the process of the nonstationary time series online prediction, a new online sequential learning algorithm with sparse update and adaptive regularization scheme is proposed based on kernel-based incremental extreme learning machine (KB-IELM). For online sparsification, a new method is presented to select sparse dictionary based on the instantaneous information measure. This method utilizes a pruning strategy, which can prune the least "significant" centers, and preserves the important ones by online minimizing the redundancy of dictionary. For adaptive regularization scheme, a new objective function is constructed based on basic ELM model. New model has different structural risks in different nonlinear regions. At each training step, new added sample could be assigned optimal regularization factor by optimization procedure. Performance comparisons of the proposed method with other existing online sequential learning methods are presented using artificial and real-word nonstationary time series data. The results indicate that the proposed method can achieve higher prediction accuracy, better generalization performance and stability. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive model predictive controller (MPC) with a function approximator is proposed to the control of the uncertain nonlinear systems. The proposed adaptive Sigmoid and Chebyshev neural networks-based MPCs (ANN-MPC and ACN-MPC) compensate the system uncertainty and control the system accurately. Using Lyapunov theory, the closed-loop signals of the linearized dynamics and the uncertainty modeling-based model predictive controller have been proved to be bounded. Accuracy of the ANN-MPC and ACN-MPC has been compared with the Runge–Kutta discretization-based nonlinear MPC on an experimental MIMO three-tank liquid-level system where a functional uncertainty is created on its dynamics. Real-time experimental results demonstrate the effectiveness of the proposed controllers. In addition, due to the faster function approximation capability of Chebyshev polynomial networks, ACN-MPC provided better control performance results. [ABSTRACT FROM AUTHOR]

Over the years, high-dimensional, noisy, and time-varying natures of the stock markets are analyzed to carry out accurate prediction. Particularly, speculators and investors are understandably eager to accurately predict stock price since millions of dollars flow through the stock markets. At this point, soft computing models have empowered them to capture the data patterns and characteristics of stock markets. However, one of the open problems in soft computing models is how to systematically determine architecture of models for given applications. In this study, Harmony Search is utilized to optimize the architecture of Neural Network, Jordan Recurrent Neural Network, Extreme Learning Machine, Recurrent Extreme Learning Machine, Generalized Linear Model, Regression Tree, and Gaussian Process Regression for 1-, 2-, 3-, 5-, 7-, and 10-day-ahead stock price prediction. The experimental results show worthy findings of stock market behavior over different prediction terms and stocks. This study also helps researchers understand which prediction model performed the best and how different conditions affect the prediction accuracy of the models. Proposed hybrid models can be successfully used by speculators and investors to make the investment or to hedge against potential risk in stock markets. [ABSTRACT FROM AUTHOR]

Driving intention has been widely used in intelligent driver assistance systems, automated driving systems, and electric vehicle control strategies. The accuracy, practicality, and timeliness of the driving intention identification model are its key issues. In this paper, a novel driver's braking intention identification model based on the Gaussian mixture-hidden Markov model (GHMM) and generalized growing and pruning radial basis function neural network (GGAP-RBFNN) is proposed to improve the identification accuracy of the model. The simplest brake pedal and vehicle speed data that are easily obtained from the vehicle are used as an observation sequence to improve practicality of the model. The data of the pressing brake pedal stage are used to identify the braking intention to improve the timeliness of the model. The experimental data collected from real vehicle tests are used for off-line training and online identification. The research results show that the accuracy of driver's braking intention identification model based on the GHMM/GGAP-RBFNN hybrid model is 94.69% for normal braking and 95.57% for slight braking, which are, respectively, 26.55% and 17.72% higher than achieved by the GHMM. In addition, the data of the pressing brake pedal stage are used for intention identification, which is 1.2 s faster than that of the existing identification model based on the GHMM. [ABSTRACT FROM AUTHOR]

Motor imagery (MI) tasks evoke event-related desynchronization (ERD) and synchronization (ERS); the ERD-/ERS-related features appearing at specific channels are frequency and time localized. Therefore, optimal channels, frequency band and time interval are of great significance for MI electroencephalography feature extraction. In this paper, channel selection method based on linear discriminant criteria is used to automatically select the channels with high discriminative powers. In addition, the concept of artificial bee colony algorithm is first introduced to find the global optimal combination of frequency band and time interval simultaneously without prior knowledge for common spatial pattern features extraction and classification. Experimental results demonstrate that this scheme can adapt to user-specific patterns and find the relatively optimal channels, frequency band and time interval for feature extraction. The classification results on the BCI Competition III Dataset IVa and BCI Competition IV Dataset IIa clearly present the effectiveness of the proposed method outperforming most of the other competing methods in the literature. [ABSTRACT FROM AUTHOR]

Recently, extreme learning machine (ELM) has attracted increasing attention due to its successful applications in classification, regression, and ranking. Normally, the desired output of the learning system using these machine learning techniques is a simple scalar output. However, there are many applications in machine learning which require more complex output rather than a simple scalar one. Therefore, structured output is used for such applications where the system is trained to predict structured output instead of simple one. Previously, support vector machine (SVM) has been introduced for structured output learning in various applications. However, from machine learning point of view, ELM is known to offer better generalization performance compared to other learning techniques. In this study, we extend ELM to more generalized framework to handle complex outputs where simple outputs are considered as special cases of it. Besides the good generalization property of ELM, the resulting model will possesses rich internal structure that reflects task-specific relations and constraints. The experimental results show that structured ELM achieves similar (for binary problems) or better (for multi-class problems) generalization performance when compared to ELM. Moreover, as verified by the simulation results, structured ELM has comparable or better precision performance with structured SVM when tested for more complex output such as object localization problem on PASCAL VOC2006. Also, the investigation on parameter selections is presented and discussed for all problems. [ABSTRACT FROM AUTHOR]

Weirs are a type of hydraulic structure used to direct and transfer water flows in the canals and overflows in the dams. The important index in computing flow discharge over the weir is discharge coefficient (Cd). The aim of this study is accurate determination of the Cd in triangular labyrinth side weirs by applying three intelligence models [i.e., artificial neural network (ANN), genetic programming (GP) and extreme learning machine (ELM)]. The calculated discharge coefficients were then compared with some experimental results. In order to examine the accuracy of Cd predictions by ANN, GP and ELM methods, five statistical indices including coefficient of determination (R2), root-mean-square error (RMSE), mean absolute percentage error (MAPE), SI and δ have been used. Results showed that R2 values in the ELM, ANN and GP methods were 0.993, 0.886 and 0.884, respectively, at training stage and 0.971, 0.965 and 0.963, respectively, at test stage. The ELM method, having MAPE, RMSE, SI and δ values of 0.81, 0.0059, 0.0082 and 0.81, respectively, at the training stage and 0.89, 0.0063, 0.0089 and 0.88, respectively, at the test stage, was superior to ANN and GP methods. The ANN model ranked next to the ELM model. [ABSTRACT FROM AUTHOR]

Accesses to physical links in Networks-on-Chip need to be appropriately arbitrated to avoid collisions. In the case of asynchronous routers, this arbitration between various clients, carrying messages with different service levels, is managed by dedicated circuits called arbiters. The latter are accustomed to allocate the shared resource to each client in a round-robin fashion; however, they may be tuned to favor certain messages more frequently by means of various digital design techniques. In this work, we make use of artificial neural networks to propose a mechanism to dynamically compute priority for each message by defining a few constraints. Based on these constraints, we first build a mathematical model for the objective function, and propose two algorithms for vector selection and resource allocation to train the artificial neural networks. We carry out a detailed comparison between seven different learning algorithms, and observe their effectiveness in terms of prediction efficiency for the application of dynamic priority arbitration. The decision is based on input parameters: available tokens, service levels, and an active request from each client. The performance of the learning algorithms has been analyzed in terms of mean squared error, true acceptance rate, number of epochs and execution time, so as to ensure mutual exclusion. [ABSTRACT FROM AUTHOR]

An optimized design of a fuzzy logic system can be regarded as setting of different parameters of the system automatically. For a single parameter, there may exist multiple feasible values. Consequently, with the increase in number of parameters, the complexity of a system increases. Type 2 fuzzy logic system has more parameters than the type 1 fuzzy logic system and is therefore much more complex than its counterpart. This paper proposes optimal parameters for an extreme learning machine-based interval type 2 fuzzy logic system to learn its best configuration. Extreme learning machine (ELM) is utilized to tune the consequent parameters of the interval type 2 fuzzy logic system (IT2FLS). A disadvantage of ELM is the random generation of its hidden neuron that causes additional uncertainty, in both approximation and learning. In order to overcome this limitation in an ELM-based IT2FLS, artificial bee colony optimization algorithm is utilized to obtain its antecedent parts parameters. The simulation results verified better performance of the proposed IT2FLS over other models with the benchmark data sets. [ABSTRACT FROM AUTHOR]

Extreme learning machine for single-hidden-layer feedforward neural networks has been extensively applied in imbalanced data learning due to its fast learning capability. Ensemble approach can effectively improve the classification performance by combining several weak learners according to a certain rule. In this paper, a novel ensemble approach on weighted extreme learning machine for imbalanced data classification problem is proposed. The weight of each base learner in the ensemble is optimized by differential evolution algorithm. Experimental results on 12 datasets show that the proposed method could achieve more classification performance compared with the simple vote-based ensemble method and non-ensemble method. [ABSTRACT FROM AUTHOR]

Single-hidden-layer feedforward neural network (SLFN) is an effective model for data classification and regression. However, it has a very important defect that it is rather time-consuming to explore the training algorithm of SLFN. In order to shorten the learning time, a special non-iterative learning algorithm was proposed, named as extreme learning machine (ELM). The main idea is that the input weights and bias are chosen randomly and the output weights are calculated by a pseudo-inverse matrix. However, ELM also has a very important drawback that it cannot achieve stable solution for different runs because of randomness. In this paper, we propose a stabilized learning algorithm based on iteration correction. The convergence analysis shows that the proposed algorithm can finish the learning process in fewer steps than the number of neurons. Three theorems and their proofs can prove that the proposed algorithm is stable. Several data sets are selected from UCI databases, and the experimental results demonstrate that the proposed algorithm is effective. [ABSTRACT FROM AUTHOR]

In this paper, an online identification method is proposed for nonlinear system identification based on extreme learning machine (ELM)-Hammerstein model. The ELM-Hammerstein model comprises an ELM neural network followed by a linear dynamic subsystem. This model is linear in parameters and nonlinear in the input. To speed up the convergence and meanwhile improve identification accuracy, a changing forgetting factor recursive least squares (CFF-RLS) method is proposed as online learning algorithm. The algorithm can identify the parameters of linear dynamic subsystem and the weights of ELM neural network simultaneously. Simulation results verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

An M-estimator-based online sequential extreme learning machine (M-OSELM) is proposed to predict chaotic time series with outliers. The M-OSELM develops from the online sequential extreme learning machine (OSELM) algorithm and retains the same excellent sequential learning ability as OSELM, but replaces the conventional least-squares cost function with a robust M-estimator-based cost function to enhance the robustness of the model to outliers. By minimizing the M-estimator-based cost function, the possible outliers are prevented from entering the model's output weights updating scheme. Meanwhile, in the sequential learning process of M-OSELM, a sequential parameter estimation approach based on error sliding window is introduced to estimate the threshold value of the M-estimator function for online outlier detection. Thanks to the built-in median operation and sliding window strategy, this approach is efficient to provide a stable estimator continuously without high computational costs, and then the potential outliers can be effectively detected. Simulation results show that the proposed M-OSELM has an excellent immunity to outliers and can always achieve better performance than its counterparts for prediction of chaotic time series when the training dataset contains outliers, ensuring at the same time all benefits of an online sequential approach. [ABSTRACT FROM AUTHOR]

The shape feature of an object represents the geometrical information which plays an important role in the image understanding and image retrieval. How to get an excellent shape feature that has rotation, scaling and translation (RST) invariance is a problem in this field. This paper proposed a novel local extreme learning machine (LELM) classification algorithm to extract the shape features. LELM finds nearest neighbors of the testing set from the original training set and trains a local classification model. The shape feature is represented by an analytic decision function with a radial basis function (RBF) kernel obtained by LELM. Our method has the following advantages: (1) LELM not only keeps the local structure of the samples, but also solves the imbalance problem between variance and bias. (2) Features obtained by the RBF kernel are RST invariant. (3) LELM is more robust against the noise and fragmentation compared to other methods. We also demonstrate the performance of the proposed method in image retrieval. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) works for generalized single-hidden-layer feedforward networks (SLFNs), and its essence is that the hidden layer of SLFNs need not be tuned. But ELM only utilizes labeled data to carry out the supervised learning task. In order to exploit unlabeled data in the ELM model, we first extend the manifold regularization (MR) framework and then demonstrate the relation between the extended MR framework and ELM. Finally, a manifold regularized extreme learning machine is derived from the proposed framework, which maintains the properties of ELM and can be applicable to large-scale learning problems. Experimental results show that the proposed semi-supervised extreme learning machine is the most cost-efficient method. It tends to have better scalability and achieve satisfactory generalization performance at a relatively faster learning speed than traditional semi-supervised learning algorithms. [ABSTRACT FROM AUTHOR]

How to predict stock price movements based on quantitative market data modeling is an attractive topic. In front of the market news and stock prices that are commonly believed as two important market data sources, how to extract and exploit the hidden information within the raw data and make both accurate and fast predictions simultaneously becomes a challenging problem. In this paper, we present the design and architecture of our trading signal mining platform that employs extreme learning machine (ELM) to make stock price prediction based on those two data sources concurrently. Comprehensive experimental comparisons between ELM and the state-of-the-art learning algorithms, including support vector machine (SVM) and back-propagation neural network (BP-NN), have been undertaken on the intra-day tick-by-tick data of the H-share market and contemporaneous news archives. The results have shown that (1) both RBF ELM and RBF SVM achieve higher prediction accuracy and faster prediction speed than BP-NN; (2) the RBF ELM achieves similar accuracy with the RBF SVM and (3) the RBF ELM has faster prediction speed than the RBF SVM. Simulations of a preliminary trading strategy with the signals are conducted. Results show that strategy with more accurate signals will make more profits with less risk. [ABSTRACT FROM AUTHOR]

Damage location detection has direct relationship with the field of aerospace structure as the detection system can inspect any exterior damage that may affect the operations of the equipment. In the literature, several kinds of learning algorithms have been applied in this field to construct the detection system and some of them gave good results. However, most learning algorithms are time-consuming due to their computational complexity so that the real-time requirement in many practical applications cannot be fulfilled. Kernel extreme learning machine (kernel ELM) is a learning algorithm, which has good prediction performance while maintaining extremely fast learning speed. Kernel ELM is originally applied to this research to predict the location of impact event on a clamped aluminum plate that simulates the shell of aerospace structures. The results were compared with several previous work, including support vector machine (SVM), and conventional back-propagation neural networks (BPNN). The comparison result reveals the effectiveness of kernel ELM for impact detection, showing that kernel ELM has comparable accuracy to SVM but much faster speed on current application than SVM and BPNN. [ABSTRACT FROM AUTHOR]