Search

Most of the existing visual object trackers are based on deep convolutional feature maps, but there have fewer works about finding new features for tracking. This paper proposes a novel tracking framework based on a full convolutional auto-encoder appearance model, which is trained by using Wasserstein distance and maximum mean discrepancy . Compared with previous works, the proposed framework has better performance in three aspects, including appearance model, update scheme, and state estimation. To address the issues of the original update scheme including poor discriminant performance under limited supervisory information, sample pollution caused by long term object occlusion, and sample importance unbalance, in this paper, a novel latent space importance weighting algorithm, a novel sample space management algorithm, and a novel IOU-based label smoothing algorithm are proposed respectively. Besides, an improved weighted loss function is adopted to address the sample imbalance issue. Finally, to improve the state estimation accuracy, the combination of Kullback-Leibler divergence and generalized intersection over union is introduced. Extensive experiments are performed on the three widely used benchmarks, and the results demonstrate the state-of-the-art performance of the proposed method. Code and models are available at https://github.com/wahahamyt/CAT.git .

In this paper, a Double Hidden Layer Neural Networks synchronization mechanism using Generative Adversarial Network (GAN) and mutual learning is used for the development of the cryptographic key exchange protocol. This protocol is used to exchange sensitive information through a public channel. At the time of exchanging sensitive information, an intruder can easily attack the vital information/signals by sniffing, spoofing, phishing, or Man-In-The-Middle attack. There is, however, hardly some investigation to investigate the randomness of the common input vector in the synchronization of two neural networks. This proposed technique provides GAN generated random input vectors for neural synchronization which are very sensitive to the seed value. To enhance the security of the synchronization process, GAN generates the best random sequence of the input vector. Two neural networks use GAN generated input vector and different random weight vector and swap their output. In some steps, it results in complete synchronization by setting the discrete weights according to the common learning rule. The synchronized weight vector serves as a session key at the end of the neural synchronization process. An increase in the weight range increases the complexity of a successful attack exponentially but the effort to build the neural key decreases over the polynomial time. The proposed technique offers synchronization and authentication steps in parallel. It is difficult for the attacker to distinguish between synchronization and authentication steps. This proposed technique has been passed through different parametric tests. The results have shown effective and robust potential. Simulations of the process show effectiveness in terms of cited results in the paper.

Gradient method is often used for the feedforward neural network training. Most of the studies so far have been focused on the square error function. In this paper, a novel entropy error function is proposed for the feedforward neural network training. The week and strong convergence analysis of the gradient method based on the entropy error function with batch input training patterns is strictly proved. Numerical examples are also given by the end of the paper for verifying the effectiveness and correctness. Compared with the square error function, our method provides both faster learning speed and better generalization for the given test problems.

This paper introduces a novel method for fuzzy modeling based on sparse Bayesian techniques. The sparse representation problems in the Takagi–Sugeno (T–S) fuzzy system identification are studied, which is to establish a T–S fuzzy system with a adaptive number of fuzzy rules and simultaneously have a minimal number of nonzero consequent parameters. The proposed method is called sparse Bayesian fuzzy inference systems (B-sparseFIS). There are two main procedures in the paper. Firstly, initial fuzzy rule of antecedent part is extracted automatically by an AP clustering method. By using the algorithm of adaptive block orthogonal matching pursuit, the number of rules is computed statistically then the main important fuzzy rules can be selected. In the algorithm, the redundant rules are eliminated for better model accuracy and generalization performance; secondly, an adaptive B-sparseFIS is exploited. The consequence of fuzzy system is identified and simplified with sparse Bayesian techniques such that many consequent parameters will approximate to zero. Four examples are provided to illustrate the effectiveness of the proposed algorithm. Furthermore, the performances of the algorithm are validated through the results of statistical analyses including parameter estimate error, MSE, NRMSE, etc.

Strip shape prediction is one of the most important technical to improve the quality of products in hot strip rolling process. In this paper, three hybrid models, including GA-MLP, MEA-MLP and PCA-MEA-MLP, are proposed for profile and flatness predictions by combining genetic algorithm (GA), mind evolutionary algorithm (MEA), principal component analysis (PCA) and multi-layer perceptron (MLP) neural networks. Mean absolute error (MAE), mean absolute percentage error, root mean squared error are adapted to evaluate the performance of the models. The results show that the data-driven model based on intelligent algorithm optimization neural networks can achieve good prediction of profile and flatness. Comparing with the hybrid GA-MLP model, the training speed of the hybrid MEA-MLP model is faster and the training time is greatly reduced. The model establishing with the input data after dimensionality reduction by PCA can reduce training time and become simple. The innovation of this paper is to propose a data-driven fast response model based on intelligent algorithm optimization neural network to replace the traditional mechanism model based on mathematical formula analysis to study complex, non-linear strip shape control in hot rolling process.

Ranging error is known to degrade significantly the target node localization accuracy. This paper investigates the use of computationally efficient positioning solution of least square (LS) in closed-form, to reduce localization accuracy loss caused by ranging error. For range-based node localization, the LS solution based on least square criterion has been confirmed to exhibit capability of optimum estimation but extensively achieve at a very complex calculation. In this paper we consider the problem how to acquire such LS solution provided with estimation performance at low complex calculation. In this paper, we use the Gauss noise model and use the weighted least squares criterion and the effective calculation method to solve the linearized equation derived from the RSS measurement, and put forward a new approach to estimate the performance of the target node location estimation. Based on the Fisher information matrix, the Cramer–Rao lower bound of target position estimation is derived based on received signal strength. We obviously indicate that the proposed algorithm can approximately achieve the LS solution in estimation performance at a markedly low complex calculation. Simulations are performed to show the improvement of the proposed algorithm.

Forecasting solar irradiation is very important to plane and size PV systems. In this paper, the fully complex valued wavelet network (FCWN) for forecasting the global solar irradiation is proposed. The complex valued gradient descent-learning algorithm is used to find the optimal complex-valued parameters of the network. An improved fully wavelet function is proposed and used as an activation function of the hidden neurons of the FCWN. The meteorological measured data of Tamanrasset city, Algeria (latitude: $$22^{\circ }48$$22ź48N; longitude: $$05^{\circ }26$$05ź26E) is used to validate the developed model. The hourly and the daily solar irradiations are forecasted using the multi input single output and the multi input multi output strategies. Several results are presented to test the feasibility and the performance of the FCWN for forecasting either daily or hourly solar irradiation. Results obtained throughout this paper show that the FCWN is a promising technique for forecasting daily and hourly solar irradiation.

This paper describes and tests an approach to improve the temporal processing capabilities of the neuroevolution of augmenting topologies (NEAT) algorithm. This algorithm is quite popular within the robotics community for the production of trained neural networks without having to determine a priori their size and topology. The main drawback of the traditional NEAT algorithm is that, even though it can implement recurrent synaptic connections, which allow it to perform some time related processing tasks, its capabilities are rather limited, especially when dealing with precise time dependent phenomena. NEAT's ability to capture the underlying dynamics that correspond to complex time series still has a lot of room for improvement. To address this issue, the paper describes a new implementation of the NEAT algorithm that is able to generate artificial neural networks (ANNs) with trainable time delayed synapses in addition to its previous capacities. We show that this approach, called $$\uptau $$?-NEAT improves the behavior of the neural networks obtained when dealing with complex time related processes. Several examples are presented, both dealing with the generation of ANNs that are able to produce complex theoretical signals such as chaotic signals or real data series, as in the case of the monthly number of international airline passengers or monthly $$\hbox {CO}_{2}$$CO2 concentrations. In these examples, $$\uptau $$?-NEAT clearly improves over the traditional NEAT algorithm in these tasks. A final example of the integration of this approach within a robot cognitive mechanism is also presented, showing the clear improvements it could provide in the modeling required for many cognitive processes.

In many applications, it is natural to use interval data to describe various kinds of uncertainties. This paper is concerned with a one-layer interval perceptron with the weights and the outputs being intervals and the inputs being real numbers. In the original learning method for this interval perceptron, an absolute value function is applied for newly learned radii of the interval weights, so as to force the radii to be positive. This approach seems unnatural somehow, and might cause oscillation in the learning procedure as indicated in our numerical experiments. In this paper, a modified learning method is proposed for this one-layer interval perceptron. We do not use the function of the absolute value, and instead, we replace, in the error function, the radius of each interval weight by a quadratic term. This simple trick does not cause any additional computational work for the learning procedure, but it brings about the following three advantages: First, the radii of the intervals of the weights are guaranteed to be positive during the learning procedure without the help of the absolute value function. Secondly, the oscillation mentioned above is eliminated and the convergence of the learning procedure is improved, as indicated by our numerical experiments. Finally, a by-product is that the convergence analysis of the learning procedure is now an easy job, while the analysis for the original learning method is at least difficult, if not impossible, due to the non-smoothness of the absolute value function involved.

Kernel methods are becoming increasingly popular for many real-world learning problems. And these methods for data analysis are frequently considered to be restricted to positive definite kernels. In practice, however, indefinite kernels arise and demand application in pattern analysis. In this paper, we present several formal extensions of kernel discriminant analysis (KDA) methods which can be used with indefinite kernels. In particular they include indefinite KDA (IKDA) based on generalized singular value decomposition (IKDA/GSVD), pseudo-inverse IKDA, null space IKDA and range space IKDA. Similar to the case of LDA-based algorithms, IKDA-based algorithms also fail to consider that different contribution of each pair of class to the discrimination. To remedy this problem, weighted schemes are incorporated into IKDA extensions in this paper and called them weighted generalized IKDA algorithms. Experiments on two real-world data sets are performed to test and evaluate the effectiveness of the proposed algorithms and the effect of weights on indefinite kernel functions. The results show that the effect of weighted schemes is very significantly.

Feedforward neural network structures have extensively been considered in the literature. In a significant volume of research and development studies hyperbolic tangent type of a neuronal nonlinearity has been utilized. This paper dwells on the widely used neuronal activation functions as well as two new ones composed of sines and cosines, and a sinc function characterizing the firing of a neuron. The viewpoint here is to consider the hidden layer(s) as transforming blocks composed of nonlinear basis functions, which may assume different forms. This paper considers 8 different activation functions which are differentiable and utilizes Levenberg-Marquardt algorithm for parameter tuning purposes. The studies carried out have a guiding quality based on empirical results on several training data sets.

This paper investigates the use of the scaled conjugate gradient (SCG) algorithm in temporal-difference (TD) learning for time series prediction. Special emphasis is given on the implementation details, after examining the theoretical background of the algorithm and the learning methodology and how these could be combined. Simple time series (linear, sinusoidal, etc.) as well as more complex ones, coming from real data, are used to examine the behavior of this novel combination of learning algorithm and methodology. Preliminary experimental results indicate that the implementation as presented in this paper indeed works, but the performance (in terms of learning speed and generalization ability) of TD learning using the SCG algorithm is not as good as expected, at least on the representative problems examined. An attempt to rationalize these results is presented.

In this paper a localized forgetting method is proposed for on-line adaptation of Gaussian radial basis function network models. It is realised that the commonly used exponential forgetting applies to the past data from the entire operating space uniformly and therefore, is not correct for nonlinear systems where dynamics are different in different operating regions. The new method proposed in this paper sets different forgetting factors in different regions according to the response of the local centre to the current measurement data. The method is applied in conjunction with the recursive orthogonal Least Squares algorithm and the computing is consequently very efficient. The developed method is applied to modelling of dissolved oxygen in a chemical reactor rig. It shows a smaller mean squared error for one-step-ahead prediction than using the uniform forgetting.

This paper presents a modification to the Minimal Resource Allocation Network (MRAN) of Yingwei et al. by introducing direct links from inputs to output and investigates its performance for noise cancellation problems. MRAN has the same structure as a Radial Basis Function network but uses a sequential learning algorithm that adds and prunes hidden neurons as input data is received sequentially so as to produce a parsimonious network. Earlier work by Sun Yonghong et al. has demonstrated the capability of MRAN to produce a compact network with excellent noise reduction properties. In this paper the capability of the direct link Minimal Resource Allocation Network (DMRAN) is evaluated by comparing it with MRAN on several nonlinear adaptive noise cancellation problems. The direct link MRAN uses the same learning algorithm as MRAN but with the introduction of direct links we are able to realise even smaller networks than MRAN with better noise reduction properties.

This paper is concerned with a proposal for a fuzzy artificial neuron with bi-nary input. The fuzzy neuron is based on fuzzy logic in that each component of the input vector is compared to a number which represent the membership value for a 0 in that position. The results of the comparisons are then combined using a generalized mean function to produce a single number which is compared to a threshold as in the case of a perceptron consisting of a linear combiner with hard limiting function. A training algorithm is developed based on an algorithm for linear inequalities described by Ho and Kashyap in a paper titled ’An Algorithm for Linear Inequalities and its Applications‘. The results obtained by simulation look promising.

Often, Neural Networks are involved in binary detectors of communication, radar or sonar systems. The design phase of a neural network detector usually requires the application of Monte Carlo trials in order to estimate some performance parameters. The classical Monte Carlo method is suitable to estimate high event probabilities (higher than 0.01), but not suitable to estimate very low event probabilities (say, 10^-5 or less). For estimations of very low false alarm probabilities (or error probabilities), a modified Monte Carlo technique, the so-called Importance Sampling (IS) technique, is considered in this paper; some topics are developed, such as optimal and suboptimal IS probability density functions (biasing density functions), control parameters and new algorithms for the minimization of the estimator error. The main novelty of this paper is the application of an efficient IS technique on neural networks, drastically reducing the number of patterns required for testing events of low probability. As a practical application, the IS technique is applied to a neural detector on a radar (or sonar) system.

Ensemble methods have attracted a wide attention, as they are learning algorithms that construct a set of classifiers and then classify new data points by taking a weighted vote of their predictions instead of creating one classifier. Random Forest is one of the most popular and powerful ensemble methods, but it suffers from some drawbacks, such as interpretability and time consumption in the prediction phase. In this paper, we introduce a new algorithm branch classification ’BrClssf’ that classifies observations using branches instead of trees, these branches are extracted from a set of randomized trees. The novelty of the proposed method is that it classifies instances according to the branch’s importance, which is defined by some criteria. This algorithm avoids the drawbacks of ensemble methods while remaining efficient. BrClssf was compared to the state-of-the-art algorithms and the results over 15 databases from the UCI Repository and Kaggle show that the BrClssf algorithm gives good performance.

In recent years, emotional neural networks (ENNs) have been extensively used in the field of time series prediction. As a variant of ENN, the radial basis emotional neural network (RBENN) is chosen as the prediction model of time series in this paper, because it has a special type of structure that can preprocess the interference in the data. However, it is difficult for many existing methods to determine network structure automatically while adjusting network parameters. To solve this problem, an RBENN based on adaptive inertia weight comprehensive learning particle swarm optimization algorithm (ADw-CLPSO-RBENN) is designed. Firstly, an adaptive inertia weight adjustment strategy based on the CLPSO algorithm (ADw-CLPSO) is exploited to balance the global and local search ability of particles. Secondly, a particle-variable dimensional learning mechanism (PVDLM) is developed based on the ADw-CLPSO algorithm, which enables particles to find the appropriate network structure while searching for the optimal parameter solution. Finally, the proposed method is evaluated in two time series and a real wastewater treatment system. The simulation results demonstrate that the proposed ADw-CLPSO-RBENN can automatically adjust to a suitable network structure, and the prediction accuracy is also better than other methods. Therefore, the proposed method has higher superiority in time series prediction.

Multi-modal semantic trajectory prediction is of great importance for location-based applications. However, predicting trajectory is not trivial facing three challenges: (1) It is difficult to integrate useful information from multi-modal and heterogeneous data in different granularity for effective feature fusion; (2) All kinds of dependencies existing in multi-modal semantic trajectories are closely coupled and dynamically evolved, forming complex dependencies for which are difficult to quantify; (3) During the model training, the distribution of each modal feature shifts in different directions, resulting in the distortion of dependencies, which is accompanied by slow convergence and inaccurate predictions. In this paper, the Complex Dependencies Auto-learning Prediction Model (CDAPM) is proposed to solve these problems. First, the effective and robust representation of each points is obtained by jointly embedding multi-modal information. Then, the dependencies attention module is proposed to calculate the dependencies weight matrix and auto-learn the contribution of each point. Also, it solves the problem of long-term dependency effectively. Position Encoding and LSTM are used to enhance the time relationship of trajectory. Finally, Mode Normalization is designed to maintain prediction accuracy by preventing the distortion of dependencies and significantly accelerate the convergence speed. Experiments on two real data sets show that CDAPM outperforms the state-of-the-art methods.

An effective financial distress prediction model is essential, as financial distress of listed companies will cause big problems for listed company owners, shareholders, banks, and regulators. Due to the special circumstances of China, Chinese listed companies that received the ‘special treatment’ label are considered to be in financial distress in this paper. We construct a relatively new and complete financial distress data set of Chinese listed companies from 2014 to 2019. Their 14 financial indicators of year $$t-2$$ are used to predict whether they will receive the ST label in year t. When building the prediction model, we first propose GA-MLP algorithm in which the Multi-Layer Perception (MLP) is optimized using Genetic Algorithm (GA) which could automatically determine the parameter values and the number of neurons in each hidden layer at the same time, and then a novel diversity-considered GA-MLP ensemble algorithm (DGAMLPE) is proposed to improve the performance. We objectively test the proposed algorithms on the constructed financial distress data set and five other datasets, and compare them with other 7 algorithms in terms of accuracy, F1-score, and AUC. Experimental results show that DGAMLPE performs very well on the financial distress data set, and it outperform other tested algorithms for the data set tested.

Improving time series forecasting accuracy is an important work for decision makers. A single model applied on a data series may not obtain satisfactory prediction accuracy. Both theoretical and empirical findings have indicated that integration of linear model and nonlinear model may provide more accurate prediction than an individual linear or nonlinear model. This paper presents a hybrid modeling approach that combines a linear autoregressive (AR) model and a nonlinear deep belief network-based autoregressive (DBN-AR) model for time series forecasting. The proposed modeling approach first applies an AR model to fit time series data, and the error between the original date and the AR model forecast data as a nonlinear component is considered, and then the error is modeled by a DBN-AR model. DBaN-AR model is a modeling method for nonlinear time series, which uses a set of deep belief networks to approximate the state-dependent functional coefficients of state dependent auto-regressive model. The proposed hybrid model can overcome limitation of an individual model and obtain more general and more accurate forecasting result than some existing hybrid models. The experiment results demonstrate that the MSE of modeling residuals using the proposed hybrid model is largely reduced compared with the results of some single prediction models and some hybrid models for one-step-ahead and multistep-ahead forecast.

This paper developes a novel model-free dual heuristic dynamic programming (DHP) algorithm combined with policy iteration and least square techniques to implement optimal consensus control of discrete-time multi-agent systems. The coupled Hamilton-Jacobi-Bellman (HJB) equations are required to be solved to achieve optimal consensus control, which is generally difficult especially under the case of unknown mathematical models. To overcome above difficulties, the DHP method is carried out by reinforcement learning utilizing online collected data rather than the accurate system dynamics. First, the performance index and corresponding Bellman equation are acquired. Each agent’s value function has quadratic form. Then, a model network is employed to approximate the accurate system dynamics. The Q-function Bellman equation is obtained next. By taking the derivative of Q-function, the DHP method is applied to construct the update formula. Convergence and stability analysis of proposed algorithm are presented. Two simulation examples are provided to illustrate the validity of the proposed algorithm.

Finger vein recognition is very important in the identity authentication, but its performance is affected significantly by noise. The widely used Conditional Generative Adversarial Nets (CGAN) de-noising algorithm without accurate texture constraints is easy to damage the texture features of the image. In this paper, we propose a finger vein de-noising algorithm based on Custom Sample-Texture Conditional Generative Adversarial Nets (CS-TCGAN). The proposed algorithm effectively protects the texture features while removing noise. Firstly, the proposed algorithm uses texture loss, adversarial loss, and content loss as constraints, which lead to a better de-noising performance on finger vein image with blurred texture.Secondly, in order to avoid the checkerboard artifacts effect caused by up-sampling in de-convolution process which results in the loss of the vein information, the dimension preserving structure is adopted in the generator network to minimize this problem. Lastly, the noise distribution of finger vein images obtained in the practical application has been investigated to generate the training dataset for obtaining a de-noising model with better generalization. Specifically, the training dataset has been established by combining Poisson noise, salt/pepper noise, Gaussian noise, and speckle noise. The experimental results illustrate that the performance of the proposed algorithm is better than the traditional filtering de-noising approaches and the widely used CGAN de-noising algorithms.

In this paper, a competitive energy scheduling strategy game of N-microgrids (MGs) inside a distributed network is considered. Each microgrid (MG) aims to maximize its profit under the noncooperative game frame. The strategy-making of each MG depends on its equipment constraints, the aggregate energy supplies of all MGs, and the energy balance of supplies and demands. To solve above discussed problem, a noncooperative game with linear coupled constraints and a distributed neurodynamic algorithm are proposed to seek the generalized Nash equilibrium (GNE). Besides, the correctness and convergence of the proposed algorithm are analyzed in detail. The effectiveness and feasibility of the proposed method are also illustrated via the simulation example.

The Boltzmann machine is one of widely used neural network models used to cope with difficult combinatorial optimisation problems. It has been used to find near optimum solutions to such hard problems as graph partitioning and the Travelling Salesman problem. However, very little is known about the time complexity of solving combinatorial optimisation problems on Boltzmann machines. This issue is important because it will help us better understand the power of Boltzmann machines in dealing with hard problems. This paper studies the time complexity of maximum matching in a graph on Boltzmann machines. It is shown that some widely-used Boltzmann machines cannot find a maximum matching in average time polynomial in the number of nodes of the graph although there are conventional deterministic algorithms which solve the problem in polynomial time. On the other hand, this paper also shows that a simpler model of Boltzmann machines, with the ’temperature‘ parameter fixed at some constant, can find a near maximum matching in polynomial average time.

Structured representation is of remarkable significance in subspace clustering. However, most of the existing subspace clustering algorithms resort to single-structured representation, which may fail to fully capture the essential characteristics of data. To address this issue, a novel multi-structured representation subspace clustering algorithm called block diagonal sparse representation (BDSR) is proposed in this paper. It takes both sparse and block diagonal structured representations into account to obtain the desired affinity matrix. The unified framework is established by integrating the block diagonal prior into the original sparse subspace clustering framework and the resulting optimization problem is iteratively solved by the inexact augmented Lagrange multipliers (IALM). Extensive experiments on both synthetic and real-world datasets well demonstrate the effectiveness and efficiency of the proposed algorithm against the state-of-the-art algorithms.

With the rapid development of deep learning, the performance on facial expression recognition (FER) has been greatly improved. However, multi-view FER, where the facial information is difficult to be fully acquired due to pose deflections, remains challenging. This paper presents a generative adversarial network (GAN) based three-stage-training algorithm for multi-view FER. Specifically, the frontal-view faces are firstly adopted to train a convolutional neural network (CNN) in an efficient way. Then, the pre-trained CNN is embedded in a well-designed GAN such that the faces with different deflections can be synthesized as the frontal-view ones containing expression features. The original and synthesized faces are finally fused and employed to retrain the CNN classifier such that the multi-view FER problem is efficiently resolved. Experimental results demonstrate that the three-stage-training algorithm can achieve better performance than some existing methods.

In this paper, we propose an effective neural network algorithm to perform singular value decomposition (SVD) of a cross-correlation matrix between two data streams. Different from traditional algorithms, the newly proposed algorithm can extract not only the principal singular vectors but also the corresponding principal singular values. First, a dynamical system is obtained from the gradient flow, which is obtained from optimization of a novel information criterion. Then, based on the dynamical system, a stable neural network algorithm, which can extract the left and right principal singular vectors, is obtained. Moreover, by satisfying orthogonality instead of orthonormality, we are able to extract the normalization scale factor as the corresponding singular value. In this case, the principal singular triplet (principal singular vectors and the corresponding singular value) of the cross-correlation matrix can be extracted by using the proposed algorithm. What’s more, the proposed algorithm can also be used for multiple PSTs extraction on the basis of sequential method. Then, convergence analysis shows that the proposed algorithm converges to the stable equilibrium point with probability 1. Last, experiment results show that the proposed algorithm is fast and stable in convergence, and can also extract multiple PSTs efficiently.

We propose, in this paper, a novel technique for large Laplacian boundary deformations using estimated rotations. The introduced method is used in the domain of Region of Interest (ROI) to align features of mesh based on Multi Mother Wavelet Neural Network (MMWNN) structure found in several mother wavelet families. The wavelet network allows the alignment of the characteristic points of the original mesh towards the target mesh. The key component of our correspondence scheme is a deformation energy that penalizes geometric distortion, encourages structure preservation and simultaneously allows mesh topology changes. To ensure the design of wavelet neural network architecture, an optimization algorithm should be applied to estimate and optimize the network parameters. Therefore, we compare our approach of 3d mesh deformation using MMWNN architecture based on genetic algorithm and our approach relying on Levenberg-Marquardt Method. We also discuss the existing comparison metrics for static and deformed triangle meshes employing the two mentioned approaches. Besides, we enumerate their strengths, weaknesses and relative performance.

Ensemble approaches introduced in the Extreme Learning Machine (ELM) literature mainly come from methods that relies on data sampling procedures, under the assumption that the training data are heterogeneously enough to set up diverse base learners. To overcome this assumption, it was proposed an ELM ensemble method based on the Negative Correlation Learning (NCL) framework, called Negative Correlation Extreme Learning Machine (NCELM). This model works in two stages: i) different ELMs are generated as base learners with random weights in the hidden layer, and ii) a NCL penalty term with the information of the ensemble prediction is introduced in each ELM minimization problem, updating the base learners, iii) second step is iterated until the ensemble converges. Although this NCL ensemble method was validated by an experimental study with multiple benchmark datasets, no information was given on the conditions about this convergence. This paper mathematically presents the sufficient conditions to guarantee the global convergence of NCELM. The update of the ensemble in each iteration is defined as a contraction mapping function, and through Banach theorem, global convergence of the ensemble is proved., Jupyter Notebook associated in https://github.com/cperales/pyridge/blob/ncelm/NCELM_convergence.ipynb

This paper proposes a new multi-view classification method by taking three sample constraints into account to automatically assign large weights to important samples and small weights to unimportant samples. To do this, we first demonstrate that different samples have different contributions to the classification models, and then propose to consider sample weight, class weight, and view weight, to overcome the influence of different levels of noise. Specifically, the sample weight for every data point is obtained by penalizing an $$\ell _{2,1}$$ -norm loss on its estimation error to reduce the influence of the sample-level noise, the class weight for each class is obtained by considering the misclassification cost as well as imbalance class to overcome the influence of class-level noise, and the view weight for each view is obtained by penalizing a squared root operator on the estimation error of each view to reduce the influence of view-level noise. In particular, our proposed sample constraints can be easily embedded in previous multi-view learning models. Experimental results on simulated and real data sets showed that our proposed method was superior to the state-of-the-art classification methods in terms of classification performance of cost-sensitive learning.

This paper proposes the Mesh Neural Network (MNN), a novel architecture which allows neurons to be connected in any topology, to efficiently route information. In MNNs, information is propagated between neurons throughout a state transition function. State and error gradients are then directly computed from state updates without backward computation. The MNN architecture and the error propagation schema is formalized and derived in tensor algebra. The proposed computational model can fully supply a gradient descent process, and is potentially suitable for very large scale sparse NNs, due to its expressivity and training efficiency, with respect to NNs based on back-propagation and computational graphs.

Although research on outlier detection methods has been an investigation area for long, few of those studies relate to an Internet of Things (IoT) domain. Several critical decisions taken on daily business operations depend on various data collected over time. Therefore, it is mandatory to guarantee its correctness, integrity, and accuracy before any further processing can commence. Outliers are often assumed to be Error by most algorithms in the past, which is always attributed to faulty sensors. Hence, this assumption has been investigated and results show that outliers can be classified into Error and Event types with the support of a Non-parametric sequence-based learning algorithm. The event type outlier is majorly caused by abnormality from sensor readings, which are very important and should not be ignored. However, the non-parametric sequence approach and other existing techniques still find it elusive to detect outliers in the global search space of a large dataset. Therefore, this paper proposes an Enhanced Non-parametric sequence learning algorithm based on Ensemble Clustering Techniques to detect Event and Error outliers in large datasets. Experiments are conducted on six different datasets from the UCL repository, except one collected from a laboratory testbed, to demonstrate the robustness and effectiveness of the proposed approach over the existing techniques. The results show a remarkable performance rate of 96.653% accuracy, 94.284% precision, and 98.112% for Error outlier detection. It also performs better in Event outlier detection with 87.611% accuracy, 71.141% precision and 85.755% specificity with 1291 s execution time.

In this paper, we propose two efficient approaches of twin support vector machines (TWSVM). The first approach is to reformulate the TWSVM formulation by introducing $$L_1$$ and $$L_\infty $$ norms in the objective functions, and convert into linear programming problems termed as LTWSVM for binary classification. The second approach is to solve the primal TWSVM, and convert into completely unconstrained minimization problem. Since the objective function is convex, piecewise quadratic but not twice differentiable, we present an efficient algorithm using the generalized Newton’s method termed as GTWSVM. Computational comparisons of the proposed LTWSVM and GTWSVM on synthetic and several real-world benchmark datasets exhibits significantly better performance with remarkably less computational time in comparison to relevant baseline methods.

Estimating the optimal number of clusters (NC) is pivotal in cluster analysis. From the viewpoint of sample geometry, a novel internal clustering validity index, which is termed the between-within cluster (BWC) index, is designed in this paper. Moreover, a method is proposed to estimate the optimal NC. The BWC index improves the well-known Silhouette index. BWC validates the clustering results from a certain clustering algorithm (e.g., affinity propagation or hierarchical) and estimates the optimal NC for many kinds of data sets, including synthetic data sets, benchmark data sets, UCI data sets, gene expression data sets, and images. Theoretical analysis and experimental studies demonstrate the effectiveness and high efficiency of the new index and method.

This paper studies network-based $$H_\infty $$ filtering problem for descriptor Markovian jump systems with a novel neural network event-triggered scheme. Firstly, to save more limited communication bandwidth, a novel neural network event-triggered scheme is introduced to dynamically adjust communication bandwidth based on desired filtering performance. Secondly, an event-triggered mode-dependent $$H_\infty $$ filter is designed for descriptor Markovian jump system. By considering the network-induced delay and the event-triggered scheme, a delay system method is used to build a novel filtering error system model. By using Lyapunov function technology and free weighting method, the criteria are obtained in terms of LMIs which guarantee the filtering error system to be regular, impulse free and stochastically stable with the $$H_\infty $$ performance. Then, a co-design method is proposed for the designed filter parameters. Finally, a numerical simulation example is employed to illustrate the effectiveness, and by a compared example, we show that the number of transmitted data produced by the proposed neural network event-triggered scheme is less than those produced by traditional event-triggered scheme.

Although Convolutional Neural Networks (CNNs) are considered as being "approximately invariant" to nuisance perturbations such as image transformation, shift, scaling, and other small deformations, some existing studies show that intense noises can cause noticeable variation to CNNs’ outputs. This paper focuses on exploring a method of measuring sensitivity by observing corresponding output variation to input perturbation on CNNs. The sensitivity is statistically defined in a bottom-up way from neuron to layer, and finally to the entire CNN network. An iterative algorithm is proposed for approximating the defined sensitivity. On the basic architecture of CNNs, the theoretically computed sensitivity is verified on the MNIST database with four types of commonly used noise distributions: Gaussian, Uniform, Salt and Pepper, and Rayleigh. Experimental results show the theoretical sensitivity is on the one hand in agreement with the actual output variation what on the maps, layers or entire networks are, and on the other hand an applicable quantitative measure for robust network selection.

This paper proposes a novel NeuroEvolutionary algorithm called Enhanced Cartesian Genetic Programming evolved Artificial Neural Network (ECGPANN) as a predictor for the lost signal samples in real time. Unlike traditional Cartesian Genetic Programming evolved Artificial Neural Network (CGPANN), the proposed algorithm introduces bi-chromosomal architecture instead of single chromosome to perform parallel evolution of topology with weights and architecture. This modification makes it suitable for obtaining global optimum solutions to predict both periodic and aperiodic lost samples at run-time. Sliding Window based Multi-instance Linear Regression (SW-MLR) and Sliding Window based Multi-instance Random Forest (SW-MRF) prediction algorithms are also exploited for the reconstruction of multiple missing samples. SW-MLR and SW-MRF being trained on fixed input/output cannot be utilized for random signal loss due to dynamic nature of number of output estimations needed at run-time. ECGPANN has the flexibility to produce variable number of outputs in real-time. Experimental results demonstrates the efficacy of the ECGPANN for both single and multi-sample loss with fix periodic and aperiodic noise using sliding window technique. The SNR improvement achieved ranges from 20 to 37 dB for periodic noise and 31–44 dB for aperiodic noise with signals having 16.6–50% samples missing. ECGPANN when compared in terms of its performance with the traditional CGPANN produced 4–5% improvement in prediction accuracy on average. The proposed ECGAPNN model is able to achieve a mean absolute error (MAE) of 0.051 (speech), 0.015 (guitar) and 0.038 (flute) for 16.6% lost/corrupted signals. MAE of 0.066 (speech), 0.020 (guitar) and 0.049 (flute) for 50% lost/corrupted data has been reported. The networks are trained and tested on audio speech signal and evaluated on music signals for its generality, with ECGPANN performing consistently better irrespective of the change in type of signals and demonstrated its robustness with change in number of missing samples in contrast to SW-MLR and SW-MRF. The ability to predict randomly variable number of missing samples make it applicable in real time.

Stacking is a model combination technique to improve prediction accuracy. Regularization is usually necessary in stacking because some predictions used in the model combination provide similar predictions. Cross-validation is generally used to select the regularization parameter, but it incurs a high computational cost. This paper proposes two simple low computational cost methods for selecting the regularization parameter. The effectiveness of the methods is examined in numerical experiments. Asymptotic results in a particular setting are also shown.

Due to the complexities of collected data that may contain outliers and noises, many variants of LDA and 2DLDA have been proposed to address this problem and have shown that the improved methods produce much better performance than classical LDA and 2DLDA. In this paper we propose a novel two-dimensional linear discriminant analysis method via information divergence. The proposed method applies the weighted L21 norm to learn a robust projection matrix in the image space. In the proposed model, we introduce the weights into the within-class scatter and the total scatter simultaneously, and learn the weights by imposing information divergence on the objective functions. To handle the proposed model, we resort to Dinkelbach’s extended algorithm to solve the proposed ratio minimization problem. Considering the characteristics of the subproblems, we exploit an equivalent representation of subproblems which can be solved by alternating optimization techniques where each block of variables has good optimization properties. The proposed model not only overcomes the small-sample-size problem, but also suppresses outliers by an adaptively weighted scheme with the guidance of information divergences. The experiments on several image data sets demonstrate that the classification performance of the proposed method is superior to that of some existing methods in the presence of outliers.

This paper studies the use and application of a fast method (non-iterative and instantaneous) for Feedforward Neural Networks training in which the weights of the hidden layer are assigned randomly, and the weights of the output layer are trained through a linear regression adjustment. The method solves two of the problems that are present in traditional training: training time and optimal structure. While traditional iterative training methods require long periods to train a single structure, the proposed method allows training a structure in a single step (not iterative). In this way, by scanning the number of neurons in the hidden layer, many structures are trained in a short time, and it is possible to obtain an optimal topology. A quality control criterion of the predictions is proposed based on the coefficient of determination that guarantees short times and an optimal number of hidden neurons to characterize a specific problem. The feasibility of the proposed method is tested by comparing its performance against building functions of the artificial neural networks toolbox in Matlab®, resulting superior in both approximation quality and training time. A rigorous study and analysis are performed for the regression of simulated data on two different surfaces with a specific noise and different topologies of the neural network. The resulting process time is at least 150 times shorter for proposed training than with the iterative training that Matlab uses, thus obtaining well-founded learning rules. A novel way of an amputated matrix is proposed that breaks the paradigm of the way multiple-output systems are trained and improves the quality of predictions with no detriment to training times.

Traditional clustering algorithms have some limitations, which are sensitive to noise and mostly applicable to convex data sets. To solve these problems, the paper proposes a novel algorithm combining spectral clustering with feature selection. Specifically, the loss item is marked with a root that can reduce the deviation value then improve the robustness of the model. And in the algorithm optimization, there is one parameter is represented by other known parameters, which can reduce the time of parameter adjustment. Then, the regular term $${{\ell }_{2,p}}\text {-norm}$$ is applied to reduce the influence of noise and redundant features and prevent the model from overfitting. Finally, Laplace matrix is constructed by kNN algorithm which is used to learn subspace and to preserve the local structure among samples, and the data after dimension reduction is used to spectral clustering. Experimental analysis on 10 benchmark datasets show that the proposed algorithm is more outperformed than the algorithms of the state-of-the-art.

Spectral feature selection (SFS) is superior to conventional feature selection methods in many aspects, by extra importing a graph matrix to preserve the subspace structure of data. However, the graph matrix of classical SFS that is generally constructed by original data easily outputs a suboptimal performance of feature selection because of the redundancy. To address this, this paper proposes a novel feature selection method via coupling the graph matrix learning and feature data learning into a unified framework, where both steps can be iteratively update until achieving the stable solution. We also apply a low-rank constraint to obtain the intrinsic structure of data to improve the robustness of learning model. Besides, an optimization algorithm is proposed to solve the proposed problem and to have fast convergence. Compared to classical and state-of-the-art feature selection methods, the proposed method achieved the competitive results on twelve real data sets.

Extreme learning machine (ELM) is one of the most remarkable machine learning algorithm in consequence of superior properties particularly its speed. ELM algorithm tends to have some drawbacks like instability and poor generalization performance in the presence of perturbation and multicollinearity. This paper introduces a novel algorithm based on Liu regression estimator (L-ELM) to handle these drawbacks. Different selection approaches have been used to determine the appropriate Liu biasing parameter. The new algorithm is tested against the basic ELM, RR-ELM, AUR-ELM and OP-ELM on nine well-known benchmark data sets. Statistical significance tests have been carried out. Experimental results show that L-ELM for at least one Liu biasing parameter generally outperforms basic ELM, RR-ELM, AUR-ELM and OP-ELM in terms of stability and generalization performance with a little lost of speed. Conversely, the training time of L-ELM is generally much slower than RR-ELM, AUR-ELM and OP-ELM. Consequently, the proposed algorithm can be considered a powerful alternative to avoid the loss of performance in regression studies

Chaos concept has been appealed in the recent optimization methods to achieve a convenient tradeoff between exploration and exploitation. Different chaotic maps have been considered to find out the appropriate one for the system dynamics. However, on particle swarm optimization (PSO), the usage of these maps has not been handled in an extensive manner, and the best fit one has not known yet. In this paper, ten chaotic maps are handled to reveal the best fit one for PSO, and to explore whether chaotic maps are necessary for PSO or not. Thirteen benchmark functions are used to perform a detailed evaluation at the first experiment. Chaotic PSO (CPSO) methods including different maps are tested on global function optimization. Concerning this, Gauss map based CPSO (GM-CPSO) has come to the forefront by achieving promising fitness values in all function evaluations and in comparison with the state-of-the-art methods. To test the efficiency of GM-CPSO on a different task, GM-CPSO is hybridized with neural network (NN) at the second experiment, and the epileptic seizure recognition is handled. Discrete wavelet transform (DWT) based features, GM-CPSO and NN are considered to design an efficient framework and to specify the type of electroencephalography signals. GM-CPSO-NN is compared with hybrid NNs including two state-of-the-art optimization methods so as to examine the efficiency of GM-CPSO. To accurately test the performances, twofold cross validation is realized on 11,500 instances, and four metrics [accuracy, area under ROC curve (AUC), sensitivity, specificity] are consulted for a detailed assessment beside of computational complexity analysis. In experiments, GM-CPSO including the necessary map, has provided remarkable fitness scores over the state-of-the-art optimization methods on optimization of various functions defined in different dimensions. Besides, the proposed framework including GM-CPSO-NN, has achieved remarkable performance by obtaining reliable accuracy (97.24%), AUC (95.67%), sensitivity (93.04%) and specificity (98.29%) scores, and by including less computational complexity than other algorithms. According to the results, GM-CPSO has arisen as the most convenient optimization method to be preferred in the formation of hybrid NNs. In addition to optimization and classification results, it’s seen that the detail sub-bands of DWT comprise necessary information for seizure recognition. Consequently, it’s revealed that GM-CPSO can be preferred on global function optimization for reliable convergence, and its usage can be extended to different disciplines like signal classification, pattern recognition or hybrid system design.

2D electrical resistivity inversion is a complicated nonlinear optimization problem, which is high-dimensional and non-convex. Using traditional neural networks to solve resistivity inversion problem is cost effective but suffers from trapping in local minima. In order to solve the above problem, a multi-output support vector regression (MSVR) nonlinear inversion method with limited ERI learning samples is researched in this paper, which considers the combined fitting errors of all outputs. Moreover, a Cauchy random and chaotic oscillation shuffled frog leaping algorithm is applied to optimize the RBF kernel widths and penalty coefficients of MSVR for improving the inversion accuracy and the computational efficiency. The key issues of data sets generation, data preprocessing and inversion flowchart are analyzed. The experimental results based on the synthetic and field examples demonstrated that the proposed algorithm is accurate, efficient and can be applied in practical engineering applications.

This paper is devoted to investigating the $$H_{\infty }$$ filtering problem for Markov jump neural networks with hidden-Markov mode observation and packet dropouts, in which the information regarding to the Markov state can not be completely acquired. To address this circumstance, a hidden Markov model (HMM)-based technique is established. That is employing a detector to detect the information of the Markov state and then giving an estimated signal of the Markov state for the filter design. Some $$H_{\infty }$$ performance analysis criteria for filtering error systems and the corresponding HMM-based filter design procedure are given. An improved activation function dividing method (AFDM) is presented for neural networks to reduce the conservatism of the obtained results. The superiority of the improved AFDM and the validity of obtained results are verified by an illustrative example.

In a conventional artificial neural model, the nonlinear activation function (AF) follows the weight sum operation. In this paper, the AF is placed before the connecting weights of each artificial neuron and hence modified artificial neural network (MANN) is proposed and the corresponding backpropagation (BP) learning algorithm is derived. Further, the slope of AF which is conventionally fixed during the training phase is adjusted for achieving better and faster training of the multi-layer artificial neural network (MANN). In this case, also both the weights and slope update algorithms are derived. To assess and compare the performance of these two ANN models, standard applications such as classification, nonlinear system identification (direct modeling), nonlinear channel equalization (inverse modeling) are implemented through simulation and compared with that obtained by conventional BP based MANN. The simulation results demonstrate that the adaptive slope based MANN outperforms the fixed slope as well as conventional MANN in terms of three different performance measures such as the number of iterations to converge, mean of the square of residual error and mean absolute deviation.

Memristor provides an available way to design and deploy swarm intelligence. As a typical swarm intelligence algorithm, ant colony optimization is implemented by the memristor crossbar array to make image edge detection in this paper. Firstly, a non-linear voltage-controlled memristor model with a relaxation term is proposed. Then, an improved ant colony optimization with padding strategy is designed. Thirdly, a memristor crossbar array with external control circuits is designed to implement ant colony optimization for image edge detection, which offers high device density and parallel computing. In the course of ant colony optimization based image edge detection deployed by memristor crossbar array, the threshold to generating edges can be directly chosen as the mean of the final conductance matrix. On the one hand, experiment results show that more delicate edges can be detected by proposed method compared to holistically-nested edge detection based on neural networks. On the other hand, Figure of merit of proposed method is better than that of Sobel operator.

Community detection is a critical issue in the field of complex networks. Capable of extracting inherent patterns and structures in high dimensional data, the non-negative matrix factorization (NMF) method has become one of the hottest research topics in community detection recently. However, this method has a significant drawback; most community detection methods using NMF require the number of communities to be preassigned or determined by searching for the best community structure among all candidates. To address the problem, in this paper, we use an improved density peak clustering to obtain the number of cores as the pre-defined parameter of nonnegative matrix factorization. Then we adopt nonnegative double singular value decomposition initialization which can rapidly reduce the approximation error of nonnegative matrix factorization. Finally, we compare and analyze the performance of different algorithms on artificial networks and real-world networks. Experimental results indicate that the proposed method is superior to the state-of-the-art methods.