Design of stabilized fuzzy relation-based neural networks driven to ensemble neurons/layers and multi-optimization.

In this study, a design methodology based on fuzzy relation-based neural networks for stabilized network structure is introduced to cope with over-fitting as well as multi-collinearity problems which generally appear in conventional fuzzy neural networks. The design method of the proposed self-organizing network structure provides an efficient solution to construct the stabilized Fuzzy Relation-based Neural Networks (FRNN) through a synergy of multi-techniques such as ensemble neurons/layers, L2-norm regularization, compromise technique, and multi-optimization, in order to generate the structure of the multi-layered self-organizing network designed with the aid of the learning as well as novel structural design. The overall network structure is realized with the aid of parallel network structure with newly added layers as well as effective node selection method through the combination technique of both a sum of squared coefficients (SSC) and performance index (PI) as a node selection criterion for each layer in FRNN. Ensemble neurons consist of the current inputs and the original inputs, and they serve to reduce the bias of the proposed FRNN by maintaining the information of the original inputs. Also, ensemble layers stand for the combination of the current layer and the front layer, and they play a role for control the variance of the model by taking into account the output of the previous layers. The least square error estimation (LSE)-based learning method with L2-norm regularization is used for constructing the stabilized network architecture, and their ensuring design methodologies result in alleviating the overfitting phenomenon and also enhancing the generalization ability. For the performance enhancement of FRNN directly affected by some parameters such as the number of input variables, collocation of the specific subset of input variables, the number of membership functions per each variable, and the order of polynomial in the consequent parts of the fuzzy rules, multi-particle swarm optimization (MPSO) is exploited for the effectively structural as well as parametric optimization of the proposed networks. That is, the multi-optimization helps achieve a compromise between the better generation performance and the alleviated over-fitting leading to the stabilization of the proposed multi-layered self-organizing network structure realized with the aid of synergistic multi-techniques and ensemble structures such as a) L2-norm regularization-based LSE learning, b) combination technique for effective node selection through the combination of SSC and PI, and c) a novel parallel network structure including newly added layers and node selection method. The performance of the proposed network structure is quantified by comprehensive experiments and comparative analysis. [ABSTRACT FROM AUTHOR]