Search

Recently, the polynomial-based radial basis function neural networks (P-RBFNNs) have been successfully applied to regression tasks. However, the redundant and non-linear partitioned data easily interfere with accurate partitioning of clusters completed in P-RBFNNs, affecting the regression performance of this existing model. Because the squared error is used as the cost function of the learning method, P-RBFNNs are sensitive to noise interference. In order to cope with these problems, this study introduces a double iterative learning-based polynomial based-RBFNNs (DP-RBFNNs) modeling that focuses on the formation of architectures to improve the accuracy of regression performance as well as enhance the robustness through double iterative learning as follows: a) support vector-based Gaussian kernel fuzzy c-means (SV-GKFCM) as a kind of the support vector-based kernel fuzzy clustering are designed to determine connections (weights) between the input and hidden layers of the proposed model. SV-GKFCM helps effectively reduce the number of redundant data to re-modify the partitioning of clusters in the DP-RBFNNs. In addition, the cluster centers can be accurately updated from the non-linear partitioned data with the aid of Gaussian kernel distance in SV-GKFCM; b) least absolute shrinkage deviations (LASD) as a robust estimation are designed to update connection weights between the hidden and output layers. The SAE (sum of absolute error) function in the LASD method is used as a cost function to reduce the noise interference in the procedure of weight estimation as well as enhance the robustness of the DP-RBFNNs. The superiority of the proposed model is demonstrated through the experimental results. [ABSTRACT FROM AUTHOR]

In this paper, we propose a polynomial kernel neural network classifier (PKNNC) based on the random sampling and information gain. Random sampling is used here to generate datasets for the construction of polynomial neurons located in the neural networks, while information gain is used to evaluate the importance of the input variables (viz. dataset features) of each neuron. Both random sampling and information gain stem from the concepts of well-known random forest models. Some traditional neural networks have certain limitations, such as slow convergence speed, easily falling to local optima and difficulty describing the polynomial relation between the input and output. In this regard, a general PKNNC is proposed, and it consists of three parts: the premise, conclusion, and aggregation. The method of designing the PKNNC is summarized as follows. In the premise section, random sampling and information gain are used to obtain multiple subdatasets that are passed to the aggregation part, and the conclusion part uses three types of polynomials. In the aggregation part, the least squares method (LSM) is used to estimate the parameters of polynomials. Furthermore, the particle swarm optimization (PSO) algorithm is exploited here to optimize the PKNNC. The overall optimization of the PKNNC combines structure optimization and parameter optimization. The PKNNC takes advantage of three types of polynomial kernel functions, random sampling techniques and information gain algorithms, which have a good ability to describe the higher-order nonlinear relationships between input and output variables and have high generalization and fast convergence capabilities. To evaluate the effectiveness of the PKNNC, numerical experiments are carried out on two types of data: machine learning data and face data. A comparative study illustrates that the proposed PKNNC leads to better performance than several conventional models. [ABSTRACT FROM AUTHOR]

• We propose fuzzy reinforced polynomial neural networks based on PNN architecture. • We propose fuzzy reinforced polynomial neurons (FRPNs) in the design of FRPNNs. • The proposed FRPNNs is a generalization of hybrid predictors (HPs). In the field of dynamic system identification and prediction, linear models (e.g., autoregressive models), nonlinear models (namely, neural networks models), and hybrid predictors (HPs) that are a hybridization of linear and nonlinear models have been proposed in the past. However, they are not completely free from limitations: they exhibit difficulties to describe high-order nonlinear relations between input and output variables. In this study, we propose fuzzy reinforced polynomial neural networks (FRPNNs), which are polynomial neural network architecture-based on fuzzy reinforced polynomial neurons (FRPNs) to overcome this limitation. The proposed FRPNs that consist of approximation part (AP) and compensation part (CP) arise as novel HPs. Here the CP for modeling nonlinear patterns can be regarded as forming the reinforced part for the AP that aims at capturing linear patterns, while AP is the linear polynomial neuron used in the conventional polynomial neural networks. In some sense, the overall FRPNNs are essentially generalized polynomial neural network architecture with novel HPs. The parameters considered in the design of the proposed fuzzy reinforced polynomial neural networks are optimized with the aid of the particle swarm optimization (PSO). The performance of FRPNNs is discussed involving time series and system identification datasets. Experimental results demonstrate that the proposed FRPNNs achieve at most the accuracy of 43.6% higher in comparison with the accuracy produced by some classical models reported in the literature. [ABSTRACT FROM AUTHOR]

In this study, we propose Hybrid Radial Basis Function Neural Networks (HRBFNNs) realized with the aid of fuzzy clustering method (Fuzzy C-Means, FCM) and polynomial neural networks. Fuzzy clustering used to form information granulation is employed to overcome a possible curse of dimensionality, while the polynomial neural network is utilized to build local models. Furthermore, genetic algorithm (GA) is exploited here to optimize the essential design parameters of the model (including fuzzification coefficient, the number of input polynomial fuzzy neurons (PFNs), and a collection of the specific subset of input PFNs) of the network. To reduce dimensionality of the input space, principal component analysis (PCA) is considered as a sound preprocessing vehicle. The performance of the HRBFNNs is quantified through a series of experiments, in which we use several modeling benchmarks of different levels of complexity (different number of input variables and the number of available data). A comparative analysis reveals that the proposed HRBFNNs exhibit higher accuracy in comparison to the accuracy produced by some models reported previously in the literature. [ABSTRACT FROM AUTHOR]

Abstract: Fuzzy modeling of complex systems is a challenging task, which involves important problems of dimensionality reduction and calls for various ways of improving the accuracy of modeling. The IG-FRBFNN, a hybrid architecture of the IG-FIS (Fuzzy Inference System) and FRBFNN (Fuzzy Radial Basis Function Neural Networks), is proposed to address these problems. The paper is concerned with the analysis and design of IG-FRBFNNs and their optimization by means of the Hierarchical Fair Competition-based Parallel Genetic Algorithm (HFC-PGA). In the proposed network, the membership functions of the premise part of the fuzzy rules of the IG-based FRBFNN model directly rely on the computation of the relevant distance between data points and the use of four types of polynomials such as constant, linear, quadratic and modified quadratic are considered for the consequent part of fuzzy rules. Moreover, the weighted Least Square (WLS) learning is exploited to estimate the coefficients of the polynomial forming the conclusion part of the rules. Since the performance of the IG-RBFNN model is affected by some key design parameters, such as a specific subset of input variables, the fuzzification coefficient of the FCM, the number of rules, and the order of polynomial of the consequent part of fuzzy rules, it becomes beneficial to carry out both structural as well as parametric optimization of the network. In this study, the HFC-PGA is used as a comprehensive optimization vehicle. The performance of the proposed model is illustrated by means of several representative numerical examples. [Copyright &y& Elsevier]

Abstract: In this study, we introduce a new design methodology of granular-oriented self-organizing Hybrid Fuzzy polynomial neural networks (HFPNN) that is based on multi-layer perceptron with context-based polynomial neurons (CPNs) or polynomial neurons (PNs). In contrast to the typical architectures encountered in polynomial neural networks (PNN), our main objective is to develop a design strategy of HFPNN as follows: (a) The first layer of the proposed network consists of context-based polynomial neuron (CPN). Here CPN is fully reflective of the structure encountered in numeric data which are granulated with the aid of context-based fuzzy c-means (C-FCM) clustering method. The context-based clustering supporting the design of information granules is completed in the space of the input data (input variables) while the formation of the clusters here is guided by a collection of some predefined fuzzy sets (so-called contexts) specified in the output space. (b) The proposed design procedure being applied to each layer of HFPNN leads to the selection of the preferred nodes of the network (CPNs or PNs) whose local characteristics (such as the number of contexts, the number of clusters, a collection of the specific subset of input variables, and the order of the polynomial) can be easily adjusted. These options contribute to the flexibility as well as simplicity and compactness of the resulting architecture of the network. For the evaluation of the performance of the proposed HFPNN, we use well-known machine learning data coming from the machine learning repository. [Copyright &y& Elsevier]

In this study, we introduce a new topology of radial basis function-based polynomial neural networks (RPNNs) that is based on a genetically optimized multi-layer perceptron with radial polynomial neurons (RPNs). This paper offers a comprehensive design methodology involving various mechanisms of optimization, especially fuzzy C-means (FCM) clustering and particle swarm optimization (PSO). In contrast to the typical architectures encountered in polynomial neural networks (PNNs), our main objective is to develop a topology and establish a comprehensive design strategy of RPNNs: (a) The architecture of the proposed network consists of radial polynomial neurons (RPN). These neurons are fully reflective of the structure encountered in numeric data, which are granulated with the aid of FCM clustering. RPN dwells on the concepts of a collection of radial basis function and the function-based nonlinear polynomial processing. (b) The PSO-based design procedure being applied to each layer of the RPNN leads to the selection of preferred nodes of the network whose local parameters (such as the number of input variables, a collection of the specific subset of input variables, the order of the polynomial, the number of clusters of FCM clustering, and a fuzzification coefficient of the FCM method) are properly adjusted. The performance of the RPNN is quantified through a series of experiments where we use several modeling benchmarks, namely a synthetic three-dimensional data and learning machine data (computer hardware data, abalone data, MPG data, and Boston housing data) already used in neuro-fuzzy modeling. A comparative analysis shows that the proposed RPNN exhibits higher accuracy in comparison with some previous models available in the literature. [ABSTRACT FROM AUTHOR]

Abstract: In this study, polynomial-based radial basis function neural networks are proposed as one of the functional components of the overall face recognition system. The system consists of the preprocessing and recognition module. The design methodology and resulting procedure of the proposed P-RBF NNs are presented. The structure helps construct a solution to high-dimensional pattern recognition problems. In data preprocessing part, principal component analysis (PCA) is generally used in face recognition. It is useful in reducing the dimensionality of the feature space. However, because it is concerned with the overall face image, it cannot guarantee the same classification rate when changing viewpoints. To compensate for these limitations, linear discriminant analysis (LDA) is used to enhance the separation between different classes. In this paper, we elaborate on the PCA-LDA algorithm and design an optimal P-RBF NNs for the recognition module. The proposed P-RBF NNs architecture consists of three functional modules such as the condition part, the conclusion part, and the inference part realized in terms of fuzzy “if–then” rules. In the condition part of fuzzy rules, the input space is partitioned with the use of fuzzy clustering realized by means of the Fuzzy C-Means (FCM) algorithm. In the conclusion part of rules, the connection weight is realized through three types of polynomials such as constant, linear, and quadratic. The coefficients of the P-RBF NNs model are obtained by fuzzy inference method forming the inference part of fuzzy rules. The essential design parameters (including learning rate, momentum, fuzzification coefficient, and the feature selection mechanism) of the networks are optimized by means of differential evolution (DE). The experimental results completed on benchmark face datasets – the AT&T, and Yale datasets demonstrate the effectiveness and efficiency of PCA-LDA combined algorithm compared with other algorithms such as PCA, LPP, 2D-PCA and 2D-LPP. A real time face recognition system realized in this way is also presented. [Copyright &y& Elsevier]

Abstract: In this study, we introduce a new design methodology of fuzzy radial basis function-based polynomial neural networks. In many cases, these models do not come with capabilities to deal with granular information. With this regard, fuzzy sets offer several interesting and useful opportunities. This study presents the development of fuzzy radial basis function-based neural networks augmented with virtual input variables. The performance of the proposed category of models is quantified through a series of experiments, in which we use two machine learning data sets and two publicly available software development effort data. [Copyright &y& Elsevier]

Abstract: In this study, we design polynomial-based radial basis function neural networks (P-RBF NNs) based on a fuzzy inference mechanism. The essential design parameters (including learning rate, momentum coefficient and fuzzification coefficient of the underlying clustering method) are optimized by means of the particle swarm optimization. The proposed P-RBF NNs dwell upon structural findings about training data that are expressed in terms of a partition matrix resulting from fuzzy clustering in this case being the fuzzy C-means (FCM). The network is of functional nature as the weights between the hidden layer and the output are some polynomials. The use of the polynomial weights becomes essential in capturing the nonlinear nature of data encountered in regression or classification problems. From the perspective of linguistic interpretation, the proposed network can be expressed as a collection of “if–then” fuzzy rules. The architecture of the networks discussed here embraces three functional modules reflecting the three phases of input–output mapping realized in rule-based architectures, namely condition formation, conclusion creation, and aggregation. The proposed classifier is applied to some synthetic and machine learning datasets, and its results are compared with those reported in the previous studies. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP2N2), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs). The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models. We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space. We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling. [Copyright &y& Elsevier]

In this study, we introduce and investigate a class of neural architectures of polynomial neural networks (PNNs), develop a comprehensive design methodology and carry out a series of numeric experiments. PNN is a flexible neural architecture whose structure (topology) is developed through learning. In particular, the number of layers of the PNN is not fixed in advance but is generated on the fly. In this sense, PNN is a self-organizing network. We distinguish between two kinds of SOPNN architectures, that is, (a) polynomial neuron (PN) based and (b) fuzzy polynomial neuron (FPN) based self-organizing networks. This taxonomy is based on the character of each neuron structure in the network. Each of them comes with two structures referred here to as basic and the modified topology. Moreover, for each topology of the SOPNN we identify two types that is a generic and advanced type. The essence of the design procedure of self-organizing polynomial neural networks (SOPNN) dwells on the group method of data handling (GMDH) (IEEE Trans. Systems Man and Cybernet. 12 (1971) 364). Each node of the PN based SOPNN exhibits a high level of flexibility and realizes a polynomial type of mapping (linear, quadratic, and cubic) between input and output variables. FPN based SOPNN dwells on the ideas of fuzzy rule-based computing and neural networks. Especially in FPN based SOPNN, the generic rules in the system assume the form “if A then y = P(x)” where A is a fuzzy relation in the condition space while P(x) is a polynomial forming a conclusion part of the rule. Each FPN (processing element) consists of a series of the nonlinear inference rules. The conclusion part of the rules, especially the regression polynomial uses several types of high-order polynomials such as linear, quadratic, modified quadratic, and cubic. As the premise part of the rules, both triangular and Gaussian-like membership functions are studied. Simulations involve a series of synthetic as well as experimental data used across various neurofuzzy systems. A detailed comparative analysis is included as well. [Copyright &y& Elsevier]

We propose a hybrid architecture based on a combination of fuzzy systems and polynomial neural networks. The resulting Hybrid Fuzzy Polynomial Neural Networks (HFPNN) dwells on the ideas of fuzzy rule-based computing and polynomial neural networks. The structure of the network comprises of fuzzy polynomial neurons (FPNs) forming the nodes of the first (input) layer of the HFPNN and polynomial neurons (PNs) that are located in the consecutive layers of the network. In the FPN (that forms a fuzzy inference system), the generic rules assume the form "if A then y = P(x)" where A is a fuzzy relation in the condition space while P(x) is a polynomial standing in the conclusion part of the rule. The conclusion part of the rules, especially the regression polynomial uses several types of high-order polynomials such as constant, linear, quadratic, and modified quadratic. As the premise part of the rules, both triangular and Gaussian-like membership functions are considered. Each PN of the network realizes a polynomial type of partial description (PD) of the mapping between input and out variables. HFPNN is a flexible neural architecture whose structure is based on the Group Method of Data Handling (GMDH) and developed through learning. In particular, the number of layers of the PNN is not fixed in advance but is generated in a dynamic way. The experimental part of the study involves two representative numerical examples such as chaotic time series and Box-Jenkins gas furnace data. [ABSTRACT FROM AUTHOR]