Search

Dimensionality reduction is an important and challenging task in machine learning and data mining. It can facilitate data clustering, classification and information retrieval. As an efficient technique for dimensionality reduction, feature selection is about finding a small feature subset preserving the most relevant information. In this paper, we propose a new criterion, called maximum projection and minimum redundancy feature selection, to address unsupervised learning scenarios. First, the feature selection is formalized with the use of the projection matrices and then characterized equivalently as a matrix factorization problem. Second, an iterative update algorithm and a greedy algorithm are proposed to tackle this problem. Third, kernel techniques are considered and the corresponding algorithm is also put forward. Finally, the proposed algorithms are compared with four state-of-the-art feature selection methods. Experimental results reported for six publicly datasets demonstrate the superiority of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Associative memories constructed and operating in the presence of big data offer an effective way to realize association mechanisms aimed at storing and recalling items. In this study, we develop a logic-driven model of two-level fuzzy associative memories augmented by autoencoding processing. It is composed of two functional modules. The first module of this architecture implements an efficient dimensionality reduction of the original high dimensional data with the use of an autoencoder. This helps achieve storing and completing the recall realized by a logic-oriented associative memory which constitutes the second module of the architecture. The optimization of the association matrices studied in the paper involves both gradient-based learning mechanisms and the algorithms of population-based optimization, i.e., particle swarm optimization (PSO) and differential evolution (DE). A suite of experimental studies is presented to quantify the performance of the proposed approach. Comparative studies are also conducted to show and quantify the advantages of the mechanisms of associative recall and storage augmented by the autoencoding process. [ABSTRACT FROM AUTHOR]

Abstract Fuzzy rule-based classifiers have been used in many applications showing better performance due to simple modular architectures and significant interpretability. Generally, researchers design this type of classifiers along with two incompatible directions: some of them devote to mine more rules from data for enhancing the accuracy of classifiers and others focus on reducing the number of rules for simplifying the classifier with high accuracy. However, from the perspective of practicability, the tradeoff between the accuracy of model and the number of rules including in the model should be carefully considered when designing classifiers. Further, these classifiers also encounter two evident limitations in the process of design: one is that the contribution of each sample versus individual attributions is equally important and another is that the concept of information granularity is not considered. These two limitations result in the reduction of accuracy of the ensuing classifier. To alleviate these two limitations and make compromise between the accuracy and the number of rules of formed classifier, in this study, a novel method is proposed to construct a fuzzy classifier by means of the principle of justifiable granularity with weighted data. The proposed method involves two main stages: the first stage concerns with the formation of an initial classification model. The initial model is constructed by engaging a synergy of Fuzzy C-Means clustering (FCM) and the principle of justifiable granularity with weighted data. The second stage focuses on the refinement of the initial classification model. The number of rules of the completely formed fuzzy classification model is equal to that of classes of experimental data. A series of experiments concerning synthetic datasets and ten UCI datasets are also implemented to exhibit the feasibility and effectiveness of the proposed classification method as well as reveal the impact of information granularity on resulting classification model. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP2N2), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs). The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models. We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space. We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling. [Copyright &y& Elsevier]

Incremental learning is an efficient computational paradigm of acquiring approximate knowledge of data in dynamic environment. Most of the research focuses on knowledge updating for single-label classification, whereas incremental mechanism for multi-label classification is of preliminary nature. This leads to considerable computation complexity to maintain desired performance. To address this challenge, we formulate a granular structure system (G S S). The proposed granular structure system in bottom-up way provides a systematic view on label-specific based classification. We demonstrate that the three-way selective ensemble (T S E N) model, a state-of-the-art solution for multi-label classification, is compatible with G S S in granulation. An incremental mechanism of G S S is introduced for both label-specific feature generation and optimization, and an incremental three-way selective ensemble algorithm for multiple instances immigration (I M O T S E N) is presented. Experiments completed on six datasets show that the proposed algorithm can maintain considerable classification performance while significantly accelerating the knowledge (G S S) updating. • Formulate a granular structure system for multi-label classification. • Present an incremental mechanism for multi-label based on proposed structure. • Demonstrate the efficiency and effectiveness of proposed algorithm. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence, we pay widespread attention to the high reliability and interpretability of resulting architectures and algorithms. As one of the main methods of machine learning, the "black-box" mechanism of neural networks lacks interpretability. In order to alleviate the low interpretability, based on belief rule-based inference methodology using the evidential reasoning approach (RIMER), we propose a convolutional rule inference network (CRIN), which is interpretable by involving rules in belief rule base (BRB) under various types of uncertain knowledge representation. Considering the influence of global data distribution on attribute weights, the determination of each attribute may be affected by all other attributes, based on the sigmoid activation function, we determine optimized attribute weights to make sure that each attribute weight is formed by all attribute values. The derivation functions with optimized attribute weights based on evidential reasoning theory also showed to ensure the feasibility of the following learning algorithm. For expressing the influence of local rules input on final output in the network, the network framework and learning algorithm of CRIN based on convolution strategy are proposed. The inference process in RIMER is used as the feedforward mechanism to ensure the interpretability of the network. Meanwhile, the gradient descent method is used as back propagation algorithm to adjust the parameters of rule base to establish a more reasonable BRB. The experimental results and comparative analysis demonstrate that the proposed CRIN has advantages in terms of interpretability and learning capability. This study suggests new trends for interpretable network and uncertain classification issues in the reasoning approach. [ABSTRACT FROM AUTHOR]