Search

In this article, we present a semantic semisupervised learning (Semantic SSL) approach targeted at unifying two machine-learning paradigms in a mutually beneficial way, where the classical support vector machine (SVM) learns to reveal primitive logic facts from data, while axiomatic fuzzy set (AFS) theory is utilized to exploit semantic knowledge and correct the wrongly perceived facts for improving the machine-learning model. This novel semisupervised method can easily produce interpretable semantic descriptions to outline different categories by forming a fuzzy set with semantic explanations realized on the basis of the AFS theory. Besides, it is known that disagreement-based semisupervised learning (SSL) can be viewed as an excellent schema so that a co-training approach with SVM and the AFS theory can be utilized to improve the resulting learning performance. Furthermore, an evaluation index is used to prune descriptions to deliver promising performance. Compared with other semisupervised approaches, the proposed approach can build a structure to reflect data-distributed information with unlabeled data and labeled data, so that the hidden information embedded in both labeled and unlabeled data can be sufficiently utilized and can potentially be applied to achieve good descriptions of each category. Experimental results demonstrate that this approach can offer a concise, comprehensible, and precise SSL frame, which strikes a balance between the interpretability and the accuracy. [ABSTRACT FROM AUTHOR]

The architecture of AND/OR fuzzy neural networks exhibits outstanding learning abilities and significant interpretation capabilities. However, AND/OR networks suffer from structure-related problems namely low efficiency and slow convergence of learning due to several reasons such as high dimensionality and gradient-based learning algorithms which lead to a visible computing overhead. In this paper, we present a two-phase fuzzy logic-oriented network design that is composed of AND/OR neurons. This design takes advantages of Randomized Neural Network (RNN) to achieve higher convergence while exhibiting good nonlinear approximation capabilities. A gradient–based learning algorithm is implemented in the second phase of the design to further reduce values of performance index. The quality of the proposed design and resulting architecture is quantified through the use of numeric data along with fuzzy sets (information granules). Experimental results meet the research's objectives and the proposed design methodology opens up new future directions for proceeding with more improvements. [ABSTRACT FROM AUTHOR]

Transfer learning is aimed at supporting the design of machine learning models in the target domain D t , given that the knowledge (model) has already been constructed in the source domain D s. The domains D t and D s (as well as the corresponding tasks T s and T t) are similar, yet not identical. As a result, the model transferred from D s to D t in this new environment exhibits its relevance (credibility) only to some limited extent. In this study, we develop an original approach, where we advocate that the knowledge transfer (model transfer) gives rise to a granular model where the level of information granularity associated with the produced results quantifies the relevance (quality or credibility) of the transferred model. In other words, we stress that the quality of knowledge transferred to D t becomes captured through a granular generalization of the original numeric model. The overall systematic design process is elaborated on by focusing on the development process of granular neural networks carried out on a basis of the numeric neural networks coming from D s. The key aspect of the design is to elevate the existing numeric neural network to its granular counterpart by admitting that the connections of the developed model come in the form of information granules, in particular intervals and fuzzy sets. The optimization process is guided by adjusting (optimizing) the level of information granularity being regarded as an essential design asset. The optimized performance index builds upon the descriptors of information granules commonly encountered in Granular Computing. In particular, coverage and specificity measures are treated as sound performance indicators of the quality of knowledge transfer (viz. the performance of the granular neural network expressed in the target domain). Several illustrative examples are provided to visualize the performance of the established design environment. [ABSTRACT FROM AUTHOR]

In classical data-driven machine learning methods, massive amounts of labeled data are required to build a high-performance prediction model. However, the amount of labeled data in many real-world applications is insufficient, so establishing a prediction model is impossible. Transfer learning has recently emerged as a solution to this problem. It exploits the knowledge accumulated in auxiliary domains to help construct prediction models in a target domain with inadequate training data. Most existing transfer learning methods solve classification tasks; only a few are devoted to regression problems. In addition, the current methods ignore the inherent phenomenon of information granularity in transfer learning. In this study, granular computing techniques are applied to transfer learning. Three granular fuzzy regression domain adaptation methods to determine the estimated values for a regression target are proposed to address three challenging cases in domain adaptation. The proposed granular fuzzy regression domain adaptation methods change the input and/or output space of the source domain's model using space transformation, so that the fuzzy rules are more compatible with the target data. Experiments on synthetic and real-world datasets validate the effectiveness of the proposed methods. [ABSTRACT FROM PUBLISHER]

Fuzzy cognitive maps (FCMs) are convenient and widely used architectures for modeling dynamic systems, which are characterized by a great deal of flexibility and adaptability. Several recent works in this area concern strategies for the development of FCMs. Although a few fully automated algorithms to learn these models from data have been introduced, the resulting FCMs are structurally considerably different than those developed by human experts. In particular, maps that were learned from data are much denser (with the density over 90% versus about 40% density of maps developed by humans). The sparseness of the maps is associated with their interpretability: the smaller the number of connections is, the higher is the transparency of the map. To this end, a novel learning approach, sparse real-coded genetic algorithms (SRCGAs), to learn FCMs is proposed. The method utilizes a density parameter to guide the learning toward a formation of maps of a certain predefined density. Comparative tests carried out for both synthetic and real-world data demonstrate that, given a suitable density estimate, the SRCGA method significantly outperforms other state-of-the-art learning methods. When the density estimate is unknown, the new method can be used in an automated fashion using a default value, and it is still able to produce models whose performance exceeds or is equal to the performance of the models generated by other methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new design methodology that supports the development of hybrid incremental models. These models result through an iterative process in which a parametric model and a nonparametric model are combined so that their underlying and complementary functionalities become fully exploited. The parametric component of the hybrid model captures some global relationships between the input variables and the output variable. The nonparametric model focuses on capturing local input-output relationships and thus augments the behavior of the model being formed at the global level. In the underlying design, we consider linear and quadratic regression to be a parametric model, whereas a fuzzy k-nearest neighbors model serves as the nonparametric counterpart of the overall model. Numeric results come from experiments that were carried out on some low-dimensional synthetic data sets and several machine learning data sets from the University of California-Irvine Machine Learning Repository. [ABSTRACT FROM AUTHOR]

Abstract: Ordinal feature values are totally ordered labels that can be considered as fuzzy sets. The formulation of proper fuzzy sets for ordinal labels is important for the systems that deal with the objects of mixed feature types. When a proper ordinal–numerical mapping for an ordinal feature of interest is given, proper fuzzy sets for the labels of the ordinal feature can be easily formulated. In this paper, we propose an adaptive method to learn proper ordinal–numerical mappings for ordinal features of interest from a given objects of mixed features including the ordinal features. The method starts with uniform ordinal–numerical mappings, and performs two steps iteratively. The first step computes a fuzzy partition over the given object set with the ordinal–numerical mappings. The second step learns new ordinal–numerical mappings from the new fuzzy partition in the way that the new mappings make the similarity between two ordinal labels be similar to the average similarity between the objects having the two labels, respectively. Through the alternate repetition of the two steps, both of the ordinal–numerical mappings and the clustering quality become gradually improved. The validity of the proposed method is strongly supported through the experiments with a modified fuzzy C-means clustering algorithm in which the proposed method is implemented. [Copyright &y& Elsevier]

Abstract: In this study, we concentrate on the fundamentals and essential development issues of logic-driven constructs of fuzzy neural networks. These networks, referred to as logic-oriented neural networks, constitute an interesting conceptual and computational framework that greatly benefits from the establishment of highly synergistic links between the technology of fuzzy sets (or granular computing, being more general) and neural networks. The most essential advantages of the proposed networks are twofold. First, the transparency of neural architectures becomes highly relevant when dealing with the mechanisms of efficient learning. Here the learning is augmented by the fact that domain knowledge could be easily incorporated in advance prior to any learning. This becomes possible given the compatibility between the architecture of the problem and the induced topology of the neural network. Second, once the training has been completed, the network can be easily interpreted and thus it directly translates into a series of truth-quantifiable logic expressions formed over a collection of information granules. The design process of the logic networks synergistically exploits the principles of information granulation, logic computing and underlying optimization including those biologically inspired techniques (such as particle swarm optimization, genetic algorithms and alike). We elaborate on the existing development trends, present key methodological pursuits and algorithms. In particular, we show how the logic blueprint of the networks is supported by the use of various constructs of fuzzy sets including logic operators, logic neurons, referential operators and fuzzy relational constructs. [Copyright &y& Elsevier]

Multiagent systems are inherently associated with their distributivity, which enforces a great deal of communication mechanisms. To effectively arrive at meaningful solutions in a vast array of problem-solving tasks, it becomes imperative to establish a sound machinery of reconciling findings which might form partial solutions to an overall problem. In this paper, we focus on a broad category of problems of collaborative data analysis realized by a collection of agents having access to their individual data and exchanging findings through their collaboration activities. Such problems of data analysis arise in the context of building a global view at a certain phenomenon (process) by viewing it from different perspectives (and thus engaging various collections of attributes by various agents). Our goal is to develop some interaction between the agents so that they could form an overall perspective, where the knowledge available locally is shared and reconciled. The underlying format of knowledge built by the agents is that of information granules and fuzzy sets in particular. We develop a comprehensive optimization scheme and discuss its two-phase nature in which the communication phase of the granular findings intertwines with the local optimization being realized by the agents at the level of the individual datasite and exploits the evidence collected from other sites. We show how the mechanism of fuzzy granulation realized in the form of a well-known fuzzy c-means (FCM) clustering can be augmented to support collaborative activities required by the agents. For this purpose, we introduce augmented versions of the original objective function used in the FCM and derive algorithmic details. We also discuss an issue of optimizing the strength of collaborative linkages, so that the reconciled findings attain the highest level of consistency (agreement). The presented experimental studies include some synthetic data and selected data sets coming from the Machine Learning repository. [ABSTRACT FROM AUTHOR]

Abstract: Information granules, such as e.g., fuzzy sets, capture essential knowledge about data and the key dependencies between them. Quite commonly, we may envision that information granules (fuzzy sets) have become a result of fuzzy clustering and therefore could be succinctly represented in the form of some fuzzy partition matrices. Interestingly, the same data set could be represented from various standpoints and this multifaceted view yields a collection of different partition matrices being reflective of the higher-order granular knowledge about the data. The levels of specificity of the clusters the data are organized into could be quite different—the larger the number of clusters, the more detailed insight into the structure of data becomes available. Given the granularity of the resulting constructs (rather than plain data themselves), one could view a collection of partition matrices as a certain type of a network of knowledge. Considering a variety of sources of knowledge encountered across the network, we are interested in forming consensus between them. In a nutshell, this leads to the construction of certain fuzzy partition matrices which “reconcile” the knowledge captured by the individual partition matrices. Given that the granularity of the sources of knowledge under consideration could vary quite substantially, we develop a unified optimization perspective by introducing fuzzy proximity matrices that are induced by the corresponding partition matrices. In the sequel, the optimization is realized on a basis of these proximity matrices. We offer a detailed algorithm and illustrate its performance using a series of numeric experiments. [Copyright &y& Elsevier]

In this study, we introduce and discuss a concept of an incremental granular model. In contrast to typical rule-based systems encountered in fuzzy modeling, the underlying principle exploited here is to consider a two-phase development of fuzzy models. First, we build a standard regression model which could he treated as a preliminary construct capturing the linear part of the data and in this way forming a backbone of the entire construct. Next, all modeling discrepancies are compensated by a collection of rules that become attached to the regions of the input space where the error is localized. The incremental model is constructed by building a collection of information granules through some specialized fuzzy clustering, called context-based (conditional) fuzzy C-meaas that is guided by the distribution of error of the linear part of the model. The architecture of the model is discussed along with the major algorithmic phases of its development. In particular, the issue of granularity of fuzzy sets of context and induced clusters is discussed vis-a-vis the performance of the model. Numeric studies concern some Iow-dimensionul synthetic data and several datasets coming from the machine learning repository. [ABSTRACT FROM AUTHOR]

Fuzzy neurons may have outstanding learning abilities and are endowed with significant interpretation capabilities. In this study, we are concerned with the development of logic networks composed of fuzzy neurons. The main phase of the design includes the granulation of the output space (via triangular fuzzy sets) being realized with the use of fuzzy equalization. In the sequel these fuzzy sets are used to guide the construction of a family of fuzzy sets in the input space. Further processing of the resulting fuzzy sets deals with some additional aggregation of those that are not sufficiently distinct. This helps reduce the size of the logic network. We include comprehensive experimentation and offer a thorough interpretation of the networks. Experiments concerning real-world continuous data help evaluate the network's appealing properties: transparent interpretability and practical feasibility. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 1249–1267, 2006. [ABSTRACT FROM AUTHOR]

In this study, we are concerned with genetically optimized fuzzy decision trees (G-DTs). Decision trees are fundamental architectures of machine learning, pattern recognition, and system modeling. Starting with the generic decision tree with discrete or interval-valued attributes, we develop its fuzzy set-based generalization. In this generalized structure we admit the values of the attributes that are represented by some membership functions. Such fuzzy decision trees are constructed in the setting of genetic optimization. The underlying genetic algorithm optimizes the parameters of the fuzzy sets associated with the individual nodes where they play a role of fuzzy "switches" by distributing a flow of processing completed within the tree. We discuss various forms of the fitness function that help capture the essence of the problem at hand (that could be either of classification nature when the dealing with discrete outputs or regression-like when handling a continuous output variable). We quantify a nature of the generalization of the tree by studying an optimally adjusted spreads of the membership functions located at the nodes of the decision tree. A series of experiments exploiting synthetic and machine learning data is used to illustrate the performance of the G-DTs. [ABSTRACT FROM AUTHOR]

In this paper, we present a new granular classifier in two versions (iterative and non–iterative), by adopting some ideas originating from a kind of Functional Link Artificial Neural Network and the Functional Network schemes. These two architectures are substantially the same: they both use a function basis instead of the usual activation function, but they are different for the learning algorithm. We augment them from the perspective of Granular Computing and information granules, designing a new kind of classifier and two learning algorithms, by taking into account granularity of information. The proposed classifier exhibits the advantages of the granular architectures, that is higher accuracy and transparency. We formally discuss the convergence of the iterative learning scheme. We carry out some numerical experiments using publicly available data, by comparing the results against those results produced by the state-of-the-art methods. In particular, we achieved sound results by invoking the iterative learning scheme. [ABSTRACT FROM AUTHOR]

Abstract Fuzzy rule-based classifiers have been used in many applications showing better performance due to simple modular architectures and significant interpretability. Generally, researchers design this type of classifiers along with two incompatible directions: some of them devote to mine more rules from data for enhancing the accuracy of classifiers and others focus on reducing the number of rules for simplifying the classifier with high accuracy. However, from the perspective of practicability, the tradeoff between the accuracy of model and the number of rules including in the model should be carefully considered when designing classifiers. Further, these classifiers also encounter two evident limitations in the process of design: one is that the contribution of each sample versus individual attributions is equally important and another is that the concept of information granularity is not considered. These two limitations result in the reduction of accuracy of the ensuing classifier. To alleviate these two limitations and make compromise between the accuracy and the number of rules of formed classifier, in this study, a novel method is proposed to construct a fuzzy classifier by means of the principle of justifiable granularity with weighted data. The proposed method involves two main stages: the first stage concerns with the formation of an initial classification model. The initial model is constructed by engaging a synergy of Fuzzy C-Means clustering (FCM) and the principle of justifiable granularity with weighted data. The second stage focuses on the refinement of the initial classification model. The number of rules of the completely formed fuzzy classification model is equal to that of classes of experimental data. A series of experiments concerning synthetic datasets and ten UCI datasets are also implemented to exhibit the feasibility and effectiveness of the proposed classification method as well as reveal the impact of information granularity on resulting classification model. [ABSTRACT FROM AUTHOR]

Kernel machines and rough sets are two classes of commonly exploited learning techniques. Kernel machines enhance traditional learning algorithms by bringing opportunities to deal with nonlinear classification problems, rough sets introduce a human-focused way to deal with uncertainty in learning problems. Granulation and approximation play a pivotal role in rough sets-based learning and reasoning. However, a way how to effectively generate fuzzy granules from data has not been fully studied so far. In this study, we integrate kernel functions with fuzzy rough set models and propose two types of kernelized fuzzy rough sets. Kernel functions are employed to compute the fuzzy T-equivalence relations between samples, thus generating fuzzy information granules in the approximation space. Subsequently fuzzy granules are used to approximate the classification based on the concepts of fuzzy lower and upper approximations. Based on the models of kernelized fuzzy rough sets, we extend the measures existing in classical rough sets to evaluate the approximation quality and approximation abilities of the attributes. We discuss the relationship between these measures and feature evaluation function ReliefF, and augment the ReliefF algorithm to enhance the robustness of these proposed measures. Finally, we apply these measures to evaluate and select features for classification problems. The experimental results help quantify the performance of the KFRS. [ABSTRACT FROM AUTHOR]

Abstract: The identification of a suitable clustering algorithm to partition data and assessment of the validity of the resultant partitioning are ongoing quests in unsupervised learning. In this study, a fuzzy granulation–degranulation criterion is proposed to evaluate the goodness of a fuzzy partitioning of the data. This, in turn, is used to determine the appropriate clustering algorithm suitable for a particular data set. In general, the quality of a partitioning is measured by computing the variance within it, which is a measure of compactness of the obtained partitioning. Here a new error function, which reflects how well the computed cluster centers represent the whole data set, is used as the goodness measure of the obtained partitioning. Thus a clustering algorithm, providing a good set of cluster centers which approximate well the whole data set, is considered to be the most suited. Thereafter this new fuzzy granulation–degranulation criterion is used to develop six new cluster validity indices. These indices mimic the definitions of the existing and well-known cluster validity indices, such as PBM-index, XB-index, PS-index, FS-index, K-index and SV-index, but use the new fuzzy granulation–degranulation based error function instead of cluster compactness. In order to evaluate the effectiveness of the proposed error function in correctly identifying the appropriate clustering algorithm for a particular data set, eight well-known clustering algorithms, K-means, Fuzzy C-means, GAK-means (genetic algorithm based K-means algorithm), a newly developed genetic point symmetry based clustering technique (GAPS-clustering), Average Linkage clustering algorithm, Expectation Maximization (EM) clustering algorithm, Self-Organizing Map (SOM) and Spectral clustering technique are evaluated on a set of six artificially generated and six real-life data sets. Results show that GAK-means is the most appropriate for most of the data sets used for the experiments. Thereafter the effectiveness of the proposed cluster validity indices in identifying the appropriate number of clusters automatically from different data sets are shown for above mentioned 12 data sets. For the purpose of comparison, results obtained with the original versions of the proposed cluster validity indices and results obtained by a density based clustering technique are also presented. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP2N2), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs). The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models. We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space. We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling. [Copyright &y& Elsevier]

Abstract: In this study, we introduce a new design methodology of granular-oriented self-organizing Hybrid Fuzzy polynomial neural networks (HFPNN) that is based on multi-layer perceptron with context-based polynomial neurons (CPNs) or polynomial neurons (PNs). In contrast to the typical architectures encountered in polynomial neural networks (PNN), our main objective is to develop a design strategy of HFPNN as follows: (a) The first layer of the proposed network consists of context-based polynomial neuron (CPN). Here CPN is fully reflective of the structure encountered in numeric data which are granulated with the aid of context-based fuzzy c-means (C-FCM) clustering method. The context-based clustering supporting the design of information granules is completed in the space of the input data (input variables) while the formation of the clusters here is guided by a collection of some predefined fuzzy sets (so-called contexts) specified in the output space. (b) The proposed design procedure being applied to each layer of HFPNN leads to the selection of the preferred nodes of the network (CPNs or PNs) whose local characteristics (such as the number of contexts, the number of clusters, a collection of the specific subset of input variables, and the order of the polynomial) can be easily adjusted. These options contribute to the flexibility as well as simplicity and compactness of the resulting architecture of the network. For the evaluation of the performance of the proposed HFPNN, we use well-known machine learning data coming from the machine learning repository. [Copyright &y& Elsevier]