Search

In this study, a novel promotive particle swarm optimizer with double hierarchical structures is proposed. It is inspired by successful mechanisms present in social and biological systems to make particles compete fairly. In the proposed method, the swarm is first divided into multiple independent subpopulations organized in a hierarchical promotion structure, which protects subpopulation at each hierarchy to search for the optima in parallel. A unidirectional communication strategy and a promotion operator are further implemented to allow excellent particles to be promoted from low-hierarchy subpopulations to high-hierarchy subpopulations. Furthermore, for the internal competition within each subpopulation of the hierarchical promotion structure, a hierarchical multiscale optimum controlled by a tiered architecture of particles is constructed for particles, in which each particle can synthesize a set of optima of its different scales. The hierarchical promotion structure can protect particles that just fly to promising regions and have low fitness from competing with the entire swarm. Also, the double hierarchical structures increase the diversity of searching. Numerical experiments and statistical analysis of results reported on 30 benchmark problems show that the proposed method improves the accuracy and convergence speed especially in solving complex problems when compared with several variations of particle swarm optimization. [ABSTRACT FROM AUTHOR]

In this article, a reinforced iterative fuzzy radial basis function neural networks (IFRBFNN) is introduced as an augmented FRBFNN architecture generated through the iterative refinement process of weighted fuzzy C-means (WFCM) clustering and weighted least square error estimation. It is well known that the location of the clusters have an effect on the classification performance of the FRBFNN. The underlying idea behind this article is how to define the center points of clusters based on the data distribution analysis as well as the improvement of classification performance. In a nutshell, while the fuzzy C-means clustering to define the positions of radial basis functions (RBFs) based in the unsupervised learning manner is usually used in a FRBFNN, the parameters (cluster centers and their ensuing polynomial coefficients) refinement of FRBFNN through the proposed iterative method lead to superb classification performance by relocating the positions of RBFs over the input space implied by supervised learning. The idea of the proposed approach is to relocate the centers (prototypes) of the fuzzy clusters by using WFCM clustering algorithm and to re-estimate the coefficients by weighted least square estimation with the aid of the cross-entropy loss values of data so that the classification performance is improved. The weight related to each data is determined by some auxiliary information (i.e., the modified version of the cross-entropy loss value of each data). When it comes to the estimation of the coefficients of the consequent polynomial in IFRBFNN, they are estimated by the weighted least square error estimation technique, where the weights of the coefficients are defined based on the cross-entropy loss values. The auxiliary information such as the weights for relocation of the centers of RBFs and the weights for the re-estimation of the polynomial coefficients should be defined from the viewpoint of enhancement of the classification performance. The cross-entropy loss value of each data is able to meet the necessity of the required supervision signal. Several numerical experiments are provided to demonstrate the usefulness of the proposed design method for the FRBFNN for classification problems. [ABSTRACT FROM AUTHOR]

In this article, we introduce a design methodology of reinforced fuzzy models based both on univariate analysis and multivariable analysis to cope with high-dimensional problems. This approach is aimed at reducing the design process and curbing computing overhead inherently associated with the increasing volume of data in terms of both their number and the dimensionality of the feature space. The critical features of the proposed fuzzy models are highlighted as follows: First, the essential input variables of the model are selected by running the univariable and multivariable analyses. In univariate analysis, input variables with a strong linear relationship with the output variable are selected through correlation analysis completed for each input space and output space. On the contrary, in multivariable analysis, input variables are chosen by comparing the determination coefficients obtained from the subsets of input variables. Second, according to the analysis of the input variable, we construct two different kinds of fuzzy models. The first fuzzy model comprises the design of the univariable-based fuzzy model (UFM) and its aggregation. The UFMs are made by the individual input variables selected from the univariable analysis using a correlation coefficient. The subspaces formed by correlation analysis are applied for determining the centers of the membership function (MF). The results produced by individual fuzzy models are aggregated through some t-conorms. The second fuzzy model is with the fuzzy clustering for improving the form of fuzzy space in the premise part of a fuzzy rule. To curb the dramatic increase in size of fuzzy rule in high-dimensional problems, the clustering space is employed as the fuzzy space, and the partition matrix produced by fuzzy provided the required degrees of the MF. Experimental studies include a suite of synthetic and publicly available data. The superiority of the proposed design methodology was demonstrated by using 34 publicly available datasets and also compared with the conventional models associated with the fuzzy rule-based models as well as the state-of-the-art models reported in the literature. [ABSTRACT FROM AUTHOR]

In this article, we introduce novel reinforced hybrid fuzzy rule-based neural networks (RHFNNs). This article is concerned with the development of the design methodologies of hybrid fuzzy rule-based model for constructing the network structure and enhancing its predictive abilities through the combination of inhomogeneous neurons [i.e., clustering-based polynomial neurons (CPNs) and polynomial neurons (PNs)] and tournament selection. The key points of the proposed RHFNN are enumerated as follows: The first layer of the proposed network consists of CPNs. CPN can effectively reflect the complex nonlinear structure encountered in the data space, and refine (granulate) it with the help of the clustering algorithm. Two types of CPNs including hard C-means (HCM) clustering-based polynomial neuron (HCPN) and fuzzy C-means (FCM) clustering-based polynomial neuron (FCPN) are designed. According to the type of CPN used in the first layer, RHFNN can be categorized into two types, namely, RHFNN based on HCPN (HRHFNN) and RHFNN based on FCPN (FRHFNN). We use PNs to construct the second and consecutive layers. PN can identify and approximate the nonlinear relationship among system's inputs and outputs. A tournament-based performance selection (TPS) algorithm stemming from evolutionary computation is used for selection of neuron. TPS not only ensures that the candidate nodes have sufficient fitting ability but also enhances the individual diversity in the node set and provides the abilities to generate better prediction nodes. In addition, L2-norm regularization is considered to reduce the deviation between coefficients and ameliorate overfitting as well as boost generalization ability. The performance of RHFNN is discussed through a variety of publicly available machine learning datasets. From the experimental results, we conclude that RHFNN achieves the best prediction accuracy on 13 of 15 datasets; the statistical analysis also confirms the superiority of RHFNN. [ABSTRACT FROM AUTHOR]

In this article, a reinforced fuzzy radial basis function neural network (R-FRBFNN) classifier is proposed. It focuses on the development of methodologies of reinforced architecture to improve classification accuracy and enhance the robust capability based on two learning strategies. The two learning strategies are summarized: 1) R-FRBFNN designed via support vector (SV)-based fuzzy C-means (FCM) clustering and softmax-based iterative reweighted least square (IRLS), which concentrate on improving the classification performance of R-FRBFNN; and 2) R-FRBFNN designed via SV-based FCM and softmax-based iterative quadratic programming (IQP), which focus on improving the robust abilities of the R-FRBFNN and reducing the effects of noise and outliers. The essential points of the proposed R-FRBFNN classifier are summarized as follows. a) The proposed R-FRBFNN consists of three phases: condition, conclusion, and inference. b) An SV-based FCM is considered for prioritizing the classification boundary and improving the classification performance of the proposed classifier. c) Three types of polynomials construct the conclusion phase. Two learning techniques are designed to update the coefficients of the polynomials. Softmax-based IRLS is a type of iterative learning technique based on Newton's method. Softmax-based IQP is more robust and avoids the degradation of generalization capabilities caused by outliers and noisy data. d) In the concept of reinforced architecture, SV-based FCM imposes compensation (membership degrees) on learning techniques according to the data characteristics encountered in the inference phase. Experimental results reported for benchmark data and outliers/noisy datasets demonstrate that the proposed classifier shows improved classification performance compared with other previously studied methods. [ABSTRACT FROM AUTHOR]

Neural networks (NNs) with least square error (LSE) estimation form a certain type of single hidden layer feed-forward NNs. In this class of networks, the input connections (weights) and the biases of hidden neurons are generated randomly and fixed after being generated. The output connections are estimated by the LSE method rather than the back-propagation method. The random generation of the input connection weights and the hidden biases results in the larger number of hidden neurons to assure the quality of classification performance. To reduce the number of neurons in the hidden layer while maintaining the classification performance, we apply a “divide and conquer” strategy in this article. In other words, we divide an overall input space into several subspaces by using information granulation technique (Fuzzy C-Means clustering algorithm) and determine the local decision boundaries among related subspaces. A decision boundary defined in the input space can be considered as being composed of several decision boundaries defined in subspaces that form the entire input space. For the decision boundaries defined in the subspaces, their nonlinearity becomes lower in comparison with the one being encountered when considering the entire input space. Through the weighted LSE estimation instead of using the LSE estimation method, the connections of several NNs can be estimated without interfering with each other. After estimating the weights, the decision boundaries defined in the related subspaces are merged to a single decision boundary by using fuzzy ensemble technique. Several machine learning datasets and one real world application dataset are used to evaluate and validate the proposed fuzzy ensemble classifier. Based on the experimental results, the proposed classifier shows better classification performance when compared with the performance of some selected classifiers. [ABSTRACT FROM AUTHOR]

In this paper, we propose reinforced fuzzy clustering-based ensemble neural networks (FCENNs) classifier. The objective of this paper is focused on the development of the design methodologies of ensemble neural networks classifier for constructing the network structure and enhancing the learning methods of fuzzy clustering-based neural networks through the combination of the probabilistic model and its learning mechanism. The proposed FCENNs classifier takes into consideration a cross-entropy error function to improve learning while $L_2$ -norm regularization is used to reduce overfitting as well as enhance generalization abilities. The essential points of the proposed reinforced FCENNs classifier can be enumerated as follows: First, in the proposed classifier, the cross-entropy error function is used as a cost function; to do this, a softmax function is applied to represent a categorical distribution located at the nodes of the output layer. Second, the learning mechanism is composed of two parts. First, fuzzy C-means clustering forms the connections (weights) of the hidden layer while the connections of the output layer are adjusted with the aid of the nonlinear least squares method using Newton's method-based learning. Third, $L_2$ norm-regularization is considered to avoid the degradation of generalization ability caused by overfitting. The learning mechanism similar to ridge regression is realized by adding $L_2$ penalty term to the cross-entropy error function. From the viewpoint of performance improvement achieved through the proposed novel learning method, the design methodology for the ensemble neural networks classifier is discussed and analyzed with the aid of a diversity of two-dimensional synthetic data and machine learning datasets. [ABSTRACT FROM AUTHOR]

This paper is concerned with a new design methodology of a reinforced interval type-2 fuzzy c-means (FCM) based fuzzy classifier (FC). The key point of this study is to reduce the computational complexity of type-2 fuzzy set-based models and to alleviate the deterioration of its generalization abilities through the synergistic effect of two algorithms: First, interval type-2 FCM (IT2FCM) is used in the hidden layer of the network and connections (weights) are adjusted by invoking the least squares error estimation method. Second, an L2-norm regularization is considered in the cost function to avoid the construction of the network suffering from overfitting. In more detail, the hidden layer of the proposed FC is realized by interval type-2 FCM clustering to deal with the factor of uncertainty involved in the problem. This type of clustering is realized by using two values of the fuzzification coefficient resulting in the interval type-2 membership functions. Once completing type reduction, the membership grades of IT2FCM are used as the outputs of the hidden layer. Instead of the backpropagation training, least squares estimator based learning is applied to adjust the functional connection being regarded as linear functions mapping the hidden layer to the output layer. In order to reduce potential overfitting, L2-norm regularization is taken into account. The effectiveness of the proposed classifier is analyzed with the aid of a number of machine learning datasets as well as face image datasets. Thorough comparative studies are also included. [ABSTRACT FROM AUTHOR]

The performance enhancement of system identification of various plastic materials to effectively recycle the waste plastics arises as a key issue studied here. For black plastics, which contain carbon black, one is unable to discriminate it from other materials. To facilitate the identification process, Fourier transform-infrared with attenuated total reflectance is used to carry out qualitative as well as quantitative analysis of black plastics. Since a spectrum obtained in this manner constitutes highly dimensional data, feature reduction becomes necessary to extract sound features and reduce the dimensionality of the original spectrum. In this study, three types of feature extraction techniques are considered: peak detection technique, feature extraction based on the chemical characteristics, and fuzzy transform-based feature extraction to determine sound discriminative features. In order to enhance classification process, fuzzy radial basis function neural networks classifier is constructed; these architectures of the classifiers take advantage of the hybrid technologies. Based upon experimental studies, it is shown that the proposed classification system with the feature extraction techniques exhibits superior performance over the performance reported for the already studied classifiers. [ABSTRACT FROM PUBLISHER]

In this study, we propose a concept of fuzzy wavelet polynomial neural networks (FWPNNs) based on concepts and constructs of polynomial neural networks and fuzzy wavelet neurons (FWNs). These networks exhibit a rule-based architecture while each rule in the FWN consists of the premise part and consequence part. The premise part is realized by using C-means clustering method, while the consequence part is realized by means of wavelet functions whose parameters are estimated with the aid of the least square method. In some sense, the FWPNN can be regarded as a generalized fuzzy wavelet neural network (FWNN). Unlike Gaussian membership functions that are commonly utilized to implement the premise part of the rules in typical FWNNs, C-means method is employed here to overcome a possible curse of dimensionality. Polynomial neural networks (PNNs) are used to express the nonlinearity of a complex system. Furthermore, the particle swarm optimization is used to optimize the design parameters of the proposed network. Based on the PNNs and FWNNs, the proposed FWPNNs take advantages of these two neural networks: it exhibits the abilities to describe high-order nonlinear relations between input and output variables and it is beneficial to describe models impacted by uncertainty. The proposed FWPNNs are applied for time-series prediction and regression problems (e.g., control of dynamic plants). Several well-known modeling benchmarks including regression and time series are considered to evaluate the performance of the proposed FWPNNs. A comparative analysis shows that the proposed FWPNNs result in better performance when comparing with some previous models reported in the literature. [ABSTRACT FROM PUBLISHER]