Search

Neural networks (NNs) with least square error (LSE) estimation form a certain type of single hidden layer feed-forward NNs. In this class of networks, the input connections (weights) and the biases of hidden neurons are generated randomly and fixed after being generated. The output connections are estimated by the LSE method rather than the back-propagation method. The random generation of the input connection weights and the hidden biases results in the larger number of hidden neurons to assure the quality of classification performance. To reduce the number of neurons in the hidden layer while maintaining the classification performance, we apply a “divide and conquer” strategy in this article. In other words, we divide an overall input space into several subspaces by using information granulation technique (Fuzzy C-Means clustering algorithm) and determine the local decision boundaries among related subspaces. A decision boundary defined in the input space can be considered as being composed of several decision boundaries defined in subspaces that form the entire input space. For the decision boundaries defined in the subspaces, their nonlinearity becomes lower in comparison with the one being encountered when considering the entire input space. Through the weighted LSE estimation instead of using the LSE estimation method, the connections of several NNs can be estimated without interfering with each other. After estimating the weights, the decision boundaries defined in the related subspaces are merged to a single decision boundary by using fuzzy ensemble technique. Several machine learning datasets and one real world application dataset are used to evaluate and validate the proposed fuzzy ensemble classifier. Based on the experimental results, the proposed classifier shows better classification performance when compared with the performance of some selected classifiers. [ABSTRACT FROM AUTHOR]

In this paper, we propose reinforced fuzzy clustering-based ensemble neural networks (FCENNs) classifier. The objective of this paper is focused on the development of the design methodologies of ensemble neural networks classifier for constructing the network structure and enhancing the learning methods of fuzzy clustering-based neural networks through the combination of the probabilistic model and its learning mechanism. The proposed FCENNs classifier takes into consideration a cross-entropy error function to improve learning while $L_2$ -norm regularization is used to reduce overfitting as well as enhance generalization abilities. The essential points of the proposed reinforced FCENNs classifier can be enumerated as follows: First, in the proposed classifier, the cross-entropy error function is used as a cost function; to do this, a softmax function is applied to represent a categorical distribution located at the nodes of the output layer. Second, the learning mechanism is composed of two parts. First, fuzzy C-means clustering forms the connections (weights) of the hidden layer while the connections of the output layer are adjusted with the aid of the nonlinear least squares method using Newton's method-based learning. Third, $L_2$ norm-regularization is considered to avoid the degradation of generalization ability caused by overfitting. The learning mechanism similar to ridge regression is realized by adding $L_2$ penalty term to the cross-entropy error function. From the viewpoint of performance improvement achieved through the proposed novel learning method, the design methodology for the ensemble neural networks classifier is discussed and analyzed with the aid of a diversity of two-dimensional synthetic data and machine learning datasets. [ABSTRACT FROM AUTHOR]

Abstract: In this study, we design polynomial-based radial basis function neural networks (P-RBF NNs) based on a fuzzy inference mechanism. The essential design parameters (including learning rate, momentum coefficient and fuzzification coefficient of the underlying clustering method) are optimized by means of the particle swarm optimization. The proposed P-RBF NNs dwell upon structural findings about training data that are expressed in terms of a partition matrix resulting from fuzzy clustering in this case being the fuzzy C-means (FCM). The network is of functional nature as the weights between the hidden layer and the output are some polynomials. The use of the polynomial weights becomes essential in capturing the nonlinear nature of data encountered in regression or classification problems. From the perspective of linguistic interpretation, the proposed network can be expressed as a collection of “if–then” fuzzy rules. The architecture of the networks discussed here embraces three functional modules reflecting the three phases of input–output mapping realized in rule-based architectures, namely condition formation, conclusion creation, and aggregation. The proposed classifier is applied to some synthetic and machine learning datasets, and its results are compared with those reported in the previous studies. [Copyright &y& Elsevier]