Search

A filter method of feature selection based on mutual information, called normalized mutual information feature selection (NMIFS), is presented. NMIFS is an enhancement over Battiti's MIFS, MIFS-U, and mRMR methods. The average normalized mutual information is proposed as a measure of redundancy among features. NMIFS outperformed MIFS, MIFS-U, and mRMR on several artificial and benchmark data sets without requiring a user-defined parameter. In addition, NMIFS is combined with a genetic algorithm to form a hybrid filter/wrapper method called GAMIFS. This includes an initialization procedure and a mutation operator based on NMIFS to speed up the convergence of the genetic algorithm. GAMIFS overcomes the limitations of incremental search algorithms that are unable to find dependencies between groups of features. Index Terms--Feature selection, genetic algorithms, multilayer perceptron (MLP) neural networks, normalized mutual information (MI).

This article is concerned with the problem of the number and dynamical properties of equilibria for a class of connected recurrent networks with two switching subnetworks. In this network model, parameters serve as switches that allow two subnetworks to be turned ON or OFF among different dynamic states. The two subnetworks are described by a nonlinear coupled equation with a complicated relation among network parameters. Thus, the number and dynamical properties of equilibria have been very hard to investigate. By using Sturm’s theorem, together with the geometrical properties of the network equation, we give a complete analysis of equilibria, including the existence, number, and dynamical properties. Necessary and sufficient conditions for the existence and exact number of equilibria are established. Moreover, the dynamical property of each equilibrium point is discussed without prior assumption of their locations. Finally, simulation examples are given to illustrate the theoretical results in this article. [ABSTRACT FROM AUTHOR]

Intermittency of electrical power in developing countries, as well as some European countries such as Turkey, can be eluded by taking advantage of solar energy. Correct prediction of solar radiation constitutes a very important step to take advantage of PV solar panels. We propose an experimental study to predict the amount of solar radiation using a classical artificial neural network (ANN) and deep learning methods. PV panel and solar radiation data were collected at Duzce University in Turkey. Moreover, we included meteorological data collected from the Meteorological Ministry of Turkey in Duzce. Data were collected on a daily basis with a 5-min interval. Data were cleaned and preprocessed to train long-short-term memory (LSTM) and ANN models to predict the solar radiation amount of one day ahead. Models were evaluated using coefficient of determination (R2), mean square error (MSE), root mean squared error (RMSE), mean absolute error (MAE), and mean biased error (MBE). LSTM outperformed ANN with R2, MSE, RMSE, MAE, and MBE of 0.93, 0.008, 0.089, 0.17, and 0.09, respectively. Moreover, we compared our results with two similar studies in the literature. The proposed study paves the way for utilizing renewable energy by leveraging the usage of PV panels. [ABSTRACT FROM AUTHOR]

A multilayer neural network based on multivalued neurons (MLMVN) is a neural network with a traditional feedforward architecture. At the same time, this network has a number of specific different features. Its backpropagation learning algorithm is derivative-free. The functionality of MLMVN is superior to that of the traditional feedforward neural networks and of a variety kernel-based networks. Its higher flexibility and faster adaptation to the target mapping enables to model complex problems using simpler networks. In this paper, the MLMVN is used to identify both type and parameters of the point spread function, whose precise identification is of crucial importance for the image deblurring. The simulation results show the high efficiency of the proposed approach. It is confirmed that the MLMVN is a powerful tool for solving classification problems, especially multiclass ones. Index Terms--Blind deconvolution, complex-valued neuron, derivative-free learning, feedforward network, multivalued neuron.

The hierarchical fast learning artificial neural network (HieFLANN) is a clustering NN that can be initialized using statistical properties of the data set. This provides the possibility of constructing the entire network autonomously with no manual intervention. This distinguishes it from many existing networks that, though hierarchically plausible, still require manual initialization processes. The unique system of hierarchical networks begins with a reduction of the high-dimensional feature space into smaller and manageable ones. This process involves using the K-iterations fast learning artificial neural network (KFLANN) to systematically cluster a square matrix containing the Mahalanobis distances (MDs) between data set features, into homogeneous feature subspaces (HFSs). The KFLANN is used for its heuristic network initialization capabilities on a given data set and requires no supervision. Through the recurring use of the KFLANN and a second stage involving canonical correlation analysis (CCA), the HieFLANN is developed. Experimental results on several standard benchmark data sets indicate that the autonomous determination of the HFS provides a viable avenue for feasible partitioning of feature subspaces. When coupled with the network transformation process, the HieFLANN yields results showing accuracies comparable with available methods. This provides a new platform by which data sets with high-dimensional feature spaces can be systematically resolved and trained autonomously, alleviating the effects of the curse of dimensionality. Index Terms--Canonical correlation analysis (CCA), data presentation sequence sensitivity (DPSS), hierarchical neural networks (NNs), homogeneous feature subspaces, K-iterations fast learning artificial neural network (KFLANN).

Single-class minimax probability machines (MPMs) offer robust novelty detection with distribution-free worst case bounds on the probability that a pattern will fall inside the normal region. However, in practice, they are too cautious in labeling patterns as outlying and so have a high false negative rate (FNR). In this paper, we propose a more aggressive version of the single-class MPM that bounds the best case probability that a pattern will fall inside the normal region. These two MPMs can then be used together to delimit the solution space. By using the hyperplane lying in the middle of this pair of MPMs, a better compromise between false positives (FPs) and false negatives (FNs), and between recall and precision can be obtained. Experiments on the real-world data sets show encouraging results. Index Terms--Kernel methods, minimax probability machines (MPMs), novelty detection.

A new model of cellular neural networks (CNNs) with transient chaos is proposed by adding negative self-feedbacks into CNNs after transforming the dynamic equation to discrete time via Euler's method. The simulation on the single neuron model shows stable fix points, bifurcation and chaos. Hence, this new CNN model has richer and more flexible dynamics, and therefore may possess better capabilities of solving various problems, compared to the conventional CNN with only stable dynamics. Index Terms--Bifurcation, cellular neural networks (CNNs), chaos.

Estimation of the dynamic spinal forces from kinematics data is very complicated because it involves the handling of the relationship between kinematic variables and electromyography (EMG) signals, as well as the relationship between EMG signals and the forces. A recurrent fuzzy neural network (RFNN) model is proposed to establish the kinematics-EMG-force relationship and model the dynamics of muscular activities. The EMG signals are used as an intermediate output and are fed back to the input layer. Since EMG is a direct reflection of muscular activities, the feedback of this model has a physical meaning. It expresses the dynamics of muscular activities in a straightforward way and takes advantage from the recurrent property. The trained model can then have the forces predicted directly from kinematic variables while bypassing the costly procedure of measuring EMG signals and avoiding the use of a biomechanics model. A learning algorithm is derived for the RFNN model. Index Terms--Fuzzy neural networks, spinal force.

Kernel methods, such as the support vector machine (SVM), are often formulated as quadratic programming (QP) problems. However, given m training patterns, a naive implementation of the QP solver takes O([m.sup.3]) training time and at least O([m.sup.2]) space. Hence, scaling up these QPs is a major stumbling block in applying kernel methods on very large data sets, and a replacement of the naive method for finding the QP solutions is highly desirable. Recently, by using approximation algorithms for the minimum enclosing ball (MEB) problem, we proposed the core vector machine (CVM) algorithm that is much faster and can handle much larger data sets than existing SVM implementations. However, the CVM can only be used with certain kernel functions and kernel methods. For example, the very popular support vector regression (SVR) cannot be used with the CVM. In this paper, we introduce the center-constrained MEB problem and subsequently extend the CVM algorithm. The generalized CVM algorithm can now be used with any linear/nonlinear kernel and can also be applied to kernel methods such as SVR and the ranking SVM. Moreover, like the original CVM, its asymptotic time complexity is again linear in m and its space complexity is independent of m. Experiments show that the generalized CVM has comparable performance with state-of-the-art SVM and SVR implementations, but is faster and produces fewer support vectors on very large data sets. Index Terms--Approximation algorithms, core vector machines (CVMs), kernel methods, minimum enclosing ball (MEB), quadratic programming, support vector machines (SVMs).

An energy function-based autoassociative memory design method to store a given set of unipolar binary memory vectors as attractive fixed points of an asynchronous discrete Hopfield network (DHN) is presented. The discrete quadratic energy function whose local minima correspond to the attractive fixed points of the network is constructed via solving a system of linear inequalities derived from the strict local minimality conditions. The weights and the thresholds are then calculated using this energy function. If the inequality system is infeasible, we conclude that no such asynchronous DHN exists, and extend the method to design a discrete piecewise quadratic energy function, which can be minimized by a generalized version of the conventional DHN, also proposed herein. In spite of its computational complexity, computer simulations indicate that the original method performs better than the conventional design methods in the sense that the memory can store, and provide the attractiveness for almost all memory sets whose cardinality is less than or equal to the dimension of its elements. The overall method, together with its extension, guarantees the storage of an arbitrary collection of memory vectors, which are mutually at least two Hamming distances away from each other, in the resulting network. Index Terms--Associative memory, memory storage and retrieval.

The olfactory system is a very efficient biological setup capable of odor information processing with neural signals. The nature of neural signals restricts the information representation to multidimensional temporal sequences of spikes. The information is contained in the interspike intervals within each individual neural signal and interspike intervals between multiple signals. A mechanism of interactions between random excitations evoked by odorants in the olfactory receptors of the epithelium and deterministic operation of the olfactory bulb is proposed in this paper. Inverse Frobenius-Perron models of the bulb's temporal sequences are fitted to the interspike distributions of temporally modulated receptor signals. Ultimately, such pattern matching results in ability to recognize odors and offer a hypothetic model for signal processing occurring in the primary stage of the olfactory system. Index Terms--Frobenius filter, interspike intervals, inverse Frobenius problem, Markov process, odorant concentration, olfactory bulb, shift map, temporal sequence.

A method to store each element of an integral memory set M [subset] [{1, 2, ..., K}.sup.n] as a fixed point into a complex-valued multistate Hopfield network is introduced. The method employs a set of inequalities to render each memory pattern as a strict local minimum of a quadratic energy landscape. Based on the solution of this system, it gives a recurrent network of n multistate neurons with complex and symmetric synaptic weights, which operates on the finite state space [{1, 2, ..., K}. sup.n] to minimize this quadratic functional. Maximum number of integral vectors that can be embedded into the energy landscape of the network by this method is investigated by computer experiments. This paper also enlightens the performance of the proposed method in reconstructing noisy gray-scale images. Index Terms--Complex-valued Hopfield network, gray-scale image retrieval, linear inequalities, multistate associative memory.

We propose an embedded/integrated feature selection method based on neural networks with Group Lasso penalty. Group Lasso regularization is considered to produce sparsity on the inputs to the network, i.e., for selection of useful features. Lasso based feature selection using a multi-layer perceptron usually requires an additional set of weights, while our Group Lasso formulation does not require that. However, Group Lasso penalty is non-differentiable at the origin. This may lead to oscillations in numerical simulations and make it difficult to analyze theoretically. To address this issue, four smoothing Group Lasso penalties are introduced. A rigorous proof for the convergence of the proposed algorithm is presented under suitable assumptions. To verify the effectiveness, a three-step algorithmic architecture is adopted in implementation. Experimental results on several datasets validate the theoretical results and demonstrate the competitive performance of the proposed method. [ABSTRACT FROM AUTHOR]

Abstract A smooth group regularization method is proposed to identify and remove the redundant input nodes of feedforward neural networks, or equivalently the redundant dimensions of the input data of a given data set. This is achieved by introducing a smooth group L 1/2 regularizer with respect to the input nodes into the error function to drive some weight vectors of the input nodes to zero. The main advantage of the method is that it can remove not only the redundant nodes, but also some redundant weights of the surviving nodes. As a comparison, the L 1 regularization (Lasso) is mainly designed for removing the redundant weights, and it is not very good at removing the redundant nodes. And the group Lasso can remove the redundant nodes, but not any weight of the surviving nodes. Another advantage of the proposed method is that it uses a smooth function to replace the non-smooth absolute value function in the common L 1/2 regularizer, and thus it reduces the oscillation caused by the non-smoothness and enables us to prove the convergence properties of the proposed training algorithm. Numerical simulations are performed to illustrate the efficiency of the algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes a new approach based on the unknown input method to synthesize the observer for polynomial Takagi–Sugeno fuzzy system with uncertainties. In this paper, the upper bounds of uncertainties are not given and the effect of uncertainties is eliminated without designing an extra controller. With the aids of the noncommon Lyapunov theory and Matlab's tools of the sum-of-square, a new observer is synthesized in which the observer form is completely different from the traditional observer forms reported in previous papers. The conditions for the observer synthesis are much relaxed and the complexity of the design process is reduced. Finally, two illustrative examples are presented to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]