Search

Abstract: Ordinal feature values are totally ordered labels that can be considered as fuzzy sets. The formulation of proper fuzzy sets for ordinal labels is important for the systems that deal with the objects of mixed feature types. When a proper ordinal–numerical mapping for an ordinal feature of interest is given, proper fuzzy sets for the labels of the ordinal feature can be easily formulated. In this paper, we propose an adaptive method to learn proper ordinal–numerical mappings for ordinal features of interest from a given objects of mixed features including the ordinal features. The method starts with uniform ordinal–numerical mappings, and performs two steps iteratively. The first step computes a fuzzy partition over the given object set with the ordinal–numerical mappings. The second step learns new ordinal–numerical mappings from the new fuzzy partition in the way that the new mappings make the similarity between two ordinal labels be similar to the average similarity between the objects having the two labels, respectively. Through the alternate repetition of the two steps, both of the ordinal–numerical mappings and the clustering quality become gradually improved. The validity of the proposed method is strongly supported through the experiments with a modified fuzzy C-means clustering algorithm in which the proposed method is implemented. [Copyright &y& Elsevier]

Feature selection plays an important role in pattern recognition and machine learning. Feature evaluation and classification complexity estimation arise as key issues in the construction of selection algorithms. To estimate classification complexity in different feature subspaces, a novel feature evaluation measure, called the neighborhood decision error rate (NDER), is proposed, which is applicable to both categorical and numerical features. We first introduce a neighborhood rough-set model to divide the sample set into decision positive regions and decision boundary regions. Then, the samples that fall within decision boundary regions are further grouped into recognizable and misclassified subsets based on class probabilities that occur in neighborhoods. The percentage of misclassified samples is viewed as the estimate of classification complexity of the corresponding feature subspaces. We present a forward greedy strategy for searching the feature subset, which minimizes the NDER and, correspondingly, minimizes the classification complexity of the selected feature subset. Both theoretical and experimental comparison with other feature selection algorithms shows that the proposed algorithm is effective for discrete and continuous features, as well as their mixture. [ABSTRACT FROM AUTHOR]

Abstract: In this study, we are concerned with system modeling which involves limited data and reconciles the developed model with some previously acquired domain knowledge being captured in the format of already constructed models. Each of these previously available models was formed on a basis of extensive data sets which are not available for the current identification pursuits. To emphasize the nature of modeling being guided by the reconciliation mechanisms, we refer to this mode of identification as experience-consistent modeling. The paper presents the conceptual and algorithmic framework by focusing on regression models. By forming a certain extended form of the performance index, it is shown that the domain knowledge captured by regression models can play a similar role as a regularization component used quite commonly in system identification. Experimental results involve both synthetic low-dimensional data and selected data coming from Machine Learning repository. The data used in the experiments tackle regression models as well as classification problems (two-class classifiers). [Copyright &y& Elsevier]

In this paper, we propose an iterative algorithm for multiple regression with fuzzy variables. While using the standard least-squares criterion as a performance index, we pose the regression problem as a gradient-descent optimisation. The separation of the evaluation of the gradient and the update of the regression variables makes it possible to avoid undue complication of analytical formulae for multiple regression with fuzzy data. The origins of fuzzy input data are traced back to the fundamental concept of information granulation and an example FCM-based granulation method is proposed and illustrated by some numerical examples. The proposed multiple regression algorithm is applied to one-, three-and nine-dimensional synthetic data sets as well as the 13-dimensional Boston Housing dataset from the machine learning repository. The algorithm''s performance is illustrated by the corresponding plots of convergence of regression parameters and the values of the prediction error of the resulting regression model. General comments on the numerical complexity of the proposed algorithm are also provided. [Copyright &y& Elsevier]

Abstract: Semi-supervised (or partial) fuzzy clustering plays an important and unique role in discovering hidden structure in data realized in presence of a certain quite limited fraction of labeled patterns. The objective of this study is to investigate and quantify the effect of various distance functions (distances) on the performance of the clustering mechanisms. The underlying goal of endowing the clustering algorithms with a higher level of flexibility is done via the use of various distances. The enhancement of this character is evaluated by means of a comprehensive assessment of quality of clusters, their ensuing discrimination abilities and the accuracy of clusters themselves. In addition to the standard Euclidean distance being commonly exploited in fuzzy clustering, three more versatile and adaptive distance measures are considered such as its weighted version, a full adaptive distance, and a kernel-based distance. Using Fuzzy C-Means (FCM) coming in its generic format, we show its semi-supervised enhancements, derive detailed formulas and analyze their effectiveness. The improvements of semi-supervised clustering are empirically evaluated and numerically quantified with the use of several Machine Learning data sets. [Copyright &y& Elsevier]

Abstract Fuzzy rule-based classifiers have been used in many applications showing better performance due to simple modular architectures and significant interpretability. Generally, researchers design this type of classifiers along with two incompatible directions: some of them devote to mine more rules from data for enhancing the accuracy of classifiers and others focus on reducing the number of rules for simplifying the classifier with high accuracy. However, from the perspective of practicability, the tradeoff between the accuracy of model and the number of rules including in the model should be carefully considered when designing classifiers. Further, these classifiers also encounter two evident limitations in the process of design: one is that the contribution of each sample versus individual attributions is equally important and another is that the concept of information granularity is not considered. These two limitations result in the reduction of accuracy of the ensuing classifier. To alleviate these two limitations and make compromise between the accuracy and the number of rules of formed classifier, in this study, a novel method is proposed to construct a fuzzy classifier by means of the principle of justifiable granularity with weighted data. The proposed method involves two main stages: the first stage concerns with the formation of an initial classification model. The initial model is constructed by engaging a synergy of Fuzzy C-Means clustering (FCM) and the principle of justifiable granularity with weighted data. The second stage focuses on the refinement of the initial classification model. The number of rules of the completely formed fuzzy classification model is equal to that of classes of experimental data. A series of experiments concerning synthetic datasets and ten UCI datasets are also implemented to exhibit the feasibility and effectiveness of the proposed classification method as well as reveal the impact of information granularity on resulting classification model. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we introduce an advanced architecture of K-means clustering-based polynomial Radial Basis Function Neural Networks (p-RBF NNs) designed with the aid of Particle Swarm Optimization (PSO) and Differential Evolution (DE) and develop a comprehensive design methodology supporting their construction. The architecture of the p-RBF NNs comes as a result of a synergistic usage of the evolutionary optimization-driven hybrid tools. The connections (weights) of the proposed p-RBF NNs being of a certain functional character and are realized by considering four types of polynomials. In order to design the optimized p-RBF NNs, a prototype (center value) of each receptive field is determined by running the K-means clustering algorithm and then a prototype and a spread of the corresponding receptive field are further optimized through running Particle Swarm Optimization (PSO) and Differential Evolution (DE). The Weighted Least Square Estimation (WLSE) is used to estimate the coefficients of the polynomials (which serve as functional connections of the network). The performance of the proposed model and the comparative analysis involving models designed with the aid of PSO and DE are presented in case of a nonlinear function and two Machine Learning (ML) datasets [Copyright &y& Elsevier]

Abstract: The identification of a suitable clustering algorithm to partition data and assessment of the validity of the resultant partitioning are ongoing quests in unsupervised learning. In this study, a fuzzy granulation–degranulation criterion is proposed to evaluate the goodness of a fuzzy partitioning of the data. This, in turn, is used to determine the appropriate clustering algorithm suitable for a particular data set. In general, the quality of a partitioning is measured by computing the variance within it, which is a measure of compactness of the obtained partitioning. Here a new error function, which reflects how well the computed cluster centers represent the whole data set, is used as the goodness measure of the obtained partitioning. Thus a clustering algorithm, providing a good set of cluster centers which approximate well the whole data set, is considered to be the most suited. Thereafter this new fuzzy granulation–degranulation criterion is used to develop six new cluster validity indices. These indices mimic the definitions of the existing and well-known cluster validity indices, such as PBM-index, XB-index, PS-index, FS-index, K-index and SV-index, but use the new fuzzy granulation–degranulation based error function instead of cluster compactness. In order to evaluate the effectiveness of the proposed error function in correctly identifying the appropriate clustering algorithm for a particular data set, eight well-known clustering algorithms, K-means, Fuzzy C-means, GAK-means (genetic algorithm based K-means algorithm), a newly developed genetic point symmetry based clustering technique (GAPS-clustering), Average Linkage clustering algorithm, Expectation Maximization (EM) clustering algorithm, Self-Organizing Map (SOM) and Spectral clustering technique are evaluated on a set of six artificially generated and six real-life data sets. Results show that GAK-means is the most appropriate for most of the data sets used for the experiments. Thereafter the effectiveness of the proposed cluster validity indices in identifying the appropriate number of clusters automatically from different data sets are shown for above mentioned 12 data sets. For the purpose of comparison, results obtained with the original versions of the proposed cluster validity indices and results obtained by a density based clustering technique are also presented. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP2N2), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs). The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models. We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space. We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling. [Copyright &y& Elsevier]

Kernel machines and rough sets are two classes of commonly exploited learning techniques. Kernel machines enhance traditional learning algorithms by bringing opportunities to deal with nonlinear classification problems, rough sets introduce a human-focused way to deal with uncertainty in learning problems. Granulation and approximation play a pivotal role in rough sets-based learning and reasoning. However, a way how to effectively generate fuzzy granules from data has not been fully studied so far. In this study, we integrate kernel functions with fuzzy rough set models and propose two types of kernelized fuzzy rough sets. Kernel functions are employed to compute the fuzzy T-equivalence relations between samples, thus generating fuzzy information granules in the approximation space. Subsequently fuzzy granules are used to approximate the classification based on the concepts of fuzzy lower and upper approximations. Based on the models of kernelized fuzzy rough sets, we extend the measures existing in classical rough sets to evaluate the approximation quality and approximation abilities of the attributes. We discuss the relationship between these measures and feature evaluation function ReliefF, and augment the ReliefF algorithm to enhance the robustness of these proposed measures. Finally, we apply these measures to evaluate and select features for classification problems. The experimental results help quantify the performance of the KFRS. [ABSTRACT FROM AUTHOR]