Search

It is a common practice to adjust the number of hidden neurons in training, and the removal of neurons in neural networks plays an indispensable role in this architecture manipulation. In this paper, a succinct and unified mathematical form is upgraded to the generic case for removing neurons based on orthogonal projection and crosswise propagation in a feedforward layer with different architectures of neural networks, and further developed for several neural networks with different architectures. For a trained neural network, the method is divided into three stages. In the first stage, the output vectors of the feedforward observation layer are classified to clusters. In the second stage, the orthogonal projection is performed to locate a neuron whose output vector can be approximated by the other output vectors in the same cluster with the least information loss. In the third stage, the previous located neuron is removed and the crosswise propagation is implemented in each cluster. On accomplishment of the three stages, the neural network with the pruned architecture is retrained. If the number of clusters is one, the method is degenerated into its special case with only one neuron being removed. The applications to different architectures of neural networks with an extension to the support vector machine are exemplified. The methodology supports in theory large-scale applications of neural networks in the real world. In addition, with minor modifications, the unified method is instructive in pruning other networks as far as they have similar network structure to the ones in this paper. It is concluded that the unified pruning method in this paper equips us an effective and powerful tool to simplify the architecture in neural networks. [ABSTRACT FROM AUTHOR]

The most difficult part of an artificial neural network to implement in hardware is the nonlinear activation function. For most implementations, the function used is the hyperbolic tangent. This function has received much attention in relation to hardware implementation. Nevertheless, there is no consensus regarding the best solution. In this paper, we propose a new approach by implementing the hyperbolic tangent in hardware with a polynomial modeling of the fractional exponential part. The results in the paper then demonstrate, through the use of an example, that this solution is faster than the CORDIC algorithm, but slower than the piecewise linear solution with the same error. The advantage over the piecewise linear approach is that it uses less memory. [ABSTRACT FROM AUTHOR]

One of the main limitations of artificial neural networks (ANN) is their high inability to know in an explicit way the relations established between explanatory variables (input) and dependent variables (output). This is a major reason why they are usually called 'black boxes.' In the last few years, several methods have been proposed to assess the relative importance of each explanatory variable. Nevertheless, it has not been possible to reach a consensus on which is the best-performing method. This is largely due to the different relative importance obtained for each variable depending on the method used. This importance also varies with the designed network architecture and/or with the initial random weights used to train the ANN. This paper proposes a procedure that seeks to minimize these problems and provides consistency in the results obtained from different methods. Essentially, the idea is to work with a set of neural networks instead of a single one. The proposed procedure is validated using a database collected from a customer satisfaction survey, which was conducted on the public transport system of Granada (Spain) in 2007. The results show that, when each method is applied independently, the variable's importance rankings are similar and, in addition, coincide with the hierarchy established by researchers who have applied other techniques. [ABSTRACT FROM AUTHOR]

This paper proposes a novel steam valving controller using radial basis function (RBF) networks-based approximate model method. Approximate model method is a kind of direct linearization approach that is derived based on the approximation of the plant's input-output model via Taylor expansion. RBF networks are used to identify the plant to implement the approximate model control law. In order to improve the performance of the approximate model controller, RBF networks weights are adjusted online using BP algorithms with an adaptive learning rate. Several simulations results demonstrate the effectiveness of the proposed controller for team valving control. [ABSTRACT FROM AUTHOR]

Optimizing the traffic signal control has an essential impact on intersections efficiency in urban transportation. This paper presents a two-stage method for intersection signal timing control. First, the traffic volume is predicted using a neuro-fuzzy network called Adaptive neuro-fuzzy inference system (ANFIS). The inputs of this network include two-dimensional, hourly and daily, traffic volume correlations. In the second stage, appropriate signal cycle and optimized timing of each phase of the signal are estimated using a combination of Self Organizing and Hopfield neural networks. The energy function of the Hopfield network is based on a traffic model derived by queuing analysis. The performance of the proposed method has been evaluated for real data. The two-dimensional correlation presents superior performance compared to hourly traffic correlation. The evaluation of proposed overall method shows considerable intersection throughput improvement comparing to the results taken form Synchro software. [ABSTRACT FROM AUTHOR]

Nonlinear system identification tends to provide highly accurate models these last decades; however, the user remains interested in finding a good balance between high-accuracy models and moderate complexity. In this paper, four balanced accuracy-complexity identification model families are proposed. These models are derived, by selecting different combinations of activation functions in a dedicated neural network design presented in our previous work (Romero-Ugalde et al. in Neurocomputing 101:170-180. doi:, ). The neural network, based on a recurrent three-layer architecture, helps to reduce the number of parameters of the model after the training phase without any loss of estimation accuracy. Even if this reduction is achieved by a convenient choice of the activation functions and the initial conditions of the synaptic weights, it nevertheless leads to a wide range of models among the most encountered in the literature. To validate the proposed approach, three different systems are identified: The first one corresponds to the unavoidable Wiener-Hammerstein system proposed in SYSID2009 as a benchmark; the second system is a flexible robot arm; and the third system corresponds to an acoustic duct. [ABSTRACT FROM AUTHOR]

The use of confidence estimation techniques on neural networks outputs plays an important role when these mathematical models are applied in many practical applications. In general, the method to provide confidence estimation is dependent on the neural network architecture, but traditionally, most popular prediction interval (PI) estimation methods are only valid under strong assumptions, which are rarely satisfied in practical real problems. In this paper, we present a comparative study of local and global strategies for PI calculations and propose novel methods in both approaches to improve the predictive power of multilayer perceptron and radial basis function neural network models when the data are heterogeneous, both in density and residual variance. We apply our methods and make comparisons in a variety of simulated and real problems. [ABSTRACT FROM AUTHOR]

The information from the electricity bills of an institution such as the University of León, with several billing points, constitutes a high-dimensional data set which is quite complicated to visualize at a glance. The use of techniques for dimensionality reduction enables to obtain a two-dimensional representation of the original data set which highlights main features in data and is easier to visualize. If these techniques are combined with interpolation methods, the resulting continuous maps allow comparison and interpretation of a whole range of possible electric data sets, not only the original one. These tools allow us to generate interactive maps that can be used by untrained people to exploit and analyze the information in electricity bills, detect penalties due to a power demand excess or power factor decrease, and make decisions with regard to electricity contracts. [ABSTRACT FROM AUTHOR]

A common statistical model for paired comparisons is the Bradley–Terry model. This research re-parameterizes the Bradley–Terry model as a single-layer artificial neural network (ANN) and shows how it can be fitted using the delta rule. The ANN model is appealing because it makes using and extending the Bradley–Terry model accessible to a broader community. It also leads to natural incremental and iterative updating methods. Several extensions are presented that allow the ANN model to learn to predict the outcome of complex, uneven two-team group competitions by rating individuals—no other published model currently does this. An incremental-learning Bradley–Terry ANN yields a probability estimate within less than 5% of the actual value training over 3,379 multi-player online matches of a popular team- and objective-based first-person shooter. [ABSTRACT FROM AUTHOR]