Search

Heterogeneous domain adaptation (HDA) is expected to achieve effective knowledge transfer from a label-rich source domain to a heterogeneous target domain with scarce labeled data. Most prior HDA methods strive to align the cross-domain feature distributions by learning domain invariant representations without considering the intrinsic semantic correlations among categories, which inevitably results in the suboptimal adaptation performance across domains. Therefore, to address this issue, we propose a novel semantic correlation transfer (SCT) method for HDA, which not only matches the marginal and conditional distributions between domains to mitigate the large domain discrepancy, but also transfers the category correlation knowledge underlying the source domain to target by maximizing the pairwise class similarity across source and target. Technically, the domainwise and classwise centroids (prototypes) are first computed and aligned according to the feature embeddings. Then, based on the derived classwise prototypes, we leverage the cosine similarity of each two classes in both domains to transfer the supervised source semantic correlation knowledge among different categories to target effectively. As a result, the feature transferability and category discriminability can be simultaneously improved during the adaptation process. Comprehensive experiments and ablation studies on standard HDA tasks, such as text-to-image, image-to-image, and text-to-text, have demonstrated the superiority of our proposed SCT against several state-of-the-art HDA methods. more...

Representing data in different spaces becomes more powerful and suitable for solving downstream learning tasks. The membership degrees obtained through fuzzy C-means (FCM) clustering cannot capture data structures sufficiently, as they represent samples from a single Euclidean geometrical perspective. To address this issue, we propose a novel fuzzy clustering model guided by spectral rotation and scaling (FCSR). In FCSR, both spectral embeddings and membership degrees are considered as new representations of data. They can complement each other from different perspectives which enables the model to engage more structural properties of the data. The process of solving the problem of membership degrees not only inherits the merits of traditional FCM but also preserves data neighborhood structures revealed by the spectral decomposition based on an affinity matrix. Furthermore, to improve the adaptability and extensibility of FCSR, the projected and kernel versions of FCSR (FCSR-P and FCSR-K) are formed. We demonstrate that FCSR-P is suitable for high-dimensional scenarios and FCSR-K can improve the linear separability among data. Extensive experiments conducted on various well-known data sets illustrate the validity of the proposed ideas. more...

Graph-based clustering approaches, especially the family of spectral clustering, have been widely used in machine learning areas. The alternatives usually engage a similarity matrix that is constructed in advance or learned from a probabilistic perspective. However, unreasonable similarity matrix construction inevitably leads to performance degradation, and the sum-to-one probability constraints may make the approaches sensitive to noisy scenarios. To address these issues, the notion of typicality-aware adaptive similarity matrix learning is presented in this study. The typicality (possibility) rather than the probability of each sample being a neighbor of other samples is measured and adaptively learned. By introducing a robust balance term, the similarity between any pairs of samples is only related to the distance between them, yet it is not affected by other samples. Therefore, the impact caused by the noisy data or outliers can be alleviated, and meanwhile, the neighborhood structures can be well captured according to the joint distance between samples and their spectral embeddings. Moreover, the generated similarity matrix has block diagonal properties that are beneficial to correct clustering. Interestingly, the results optimized by the typicality-aware adaptive similarity matrix learning share the common essence with the Gaussian kernel function, and the latter can be directly derived from the former. Extensive experiments on synthetic and well-known benchmark datasets demonstrate the superiority of the proposed idea when comparing with some state-of-the-art methods. more...

Segmentation on the left atrium from magnetic resonance imaging scans is a crucial task for the clinical diagnosis of cardiac diseases. Automatic segmentation methods based on deep learning have demonstrated their advantages in recent years, but they often suffer from the difficulty to process high uncertainty of some regions. Focusing on the cognitive uncertainty in deep neural networks, we build an uncertainty-aware model based on anchor fuzzy sets in this article. A fuzzy entropy measure defined on membership functions is employed to quantify the uncertainty induced by an anchor's membership degrees to the individual categories. A key insight in this study is that the anchor regions with higher uncertainty are considered to include hard-generalized cases. Based on this prior of high uncertainty about image segmentation, the uncertainty-aware model exhibits the self-learning attention that sheds light on the regions the deep learning should focus on. In addition, our segmentation model with an uncertainty-aware plug-in unit highlights its nonlocal perception due to Fourier convolution and global self-attention mechanism. Experiments show that our model can achieve competitive results on left atrium segmentation tasks. This article provides some useful guidelines for using uncertainty modeling to improve the performance of deep learning systems for image segmentation tasks. more...

It is necessary to improve the performance of some special classes or to particularly protect them from attacks in adversarial learning. This article proposes a framework combining cost-sensitive classification and adversarial learning together to train a model that can distinguish between the protected and unprotected classes, such that the protected classes are less vulnerable to adversarial examples. We find in this framework an interesting phenomenon during the training of deep neural networks, called the Min–Max property, that is, the absolute values of most parameters in the convolutional layer approach 0 while the absolute values of a few parameters are significantly larger, becoming bigger. Based on this Min–Max property which is formulated and analyzed in a view of random distribution, we further build a new defense model against adversarial examples for adversarial robustness improvement. An advantage of the built model is that it performs better than the standard one and can combine with adversarial training to achieve improved performance. It is experimentally confirmed that, regarding the average accuracy of all classes, our model is almost as same as the existing models when an attack does not occur and is better than the existing models when an attack occurs. Specifically, regarding the accuracy of protected classes, the proposed model is much better than the existing models when an attack occurs. more...

Fine-grained visual categorization is a challenging issue owing to high intra-class and low inter-class variances. Classical approaches rely on pre-trained models or many fine annotations. In this paper, we observe that spatial and frequency information provides distinct image views, and propose a new spatial–frequency feature fusion (SFFF) perspective to handle this challenging issue. Specifically, we design a heterogeneous feature extraction loss function, construct a global and local fusion SFFF network, and propose an importance-sparsity selection strategy. For feature extraction, we focus on the frequency domain feature learning network, extract fine-grained features, and achieve feature complementarity. For feature selection, we propose importance ranking and sparse regularity to constrain spatial–frequency features. For feature fusion, we design a spatial–frequency loss and an inter-layer switching strategy to achieve local-global collaboration. Comparative experiments were performed on popular fine-grained datasets and classic datasets such as CUB200-2011, Stanford Cars, Stanford Dogs, FGVC-Aircraft, and CIFAR100. The effectiveness and outstanding performance of SFFF are confirmed by comparisons with more than 40 state-of-the-art fine-grained categorization methods. Ablation studies and visualizations are provided to facilitate an understanding of our approach. more...

Fuzzy $\beta$-covering rough sets, as an effective extension of covering-based rough sets, have been concerned by many researchers. All fuzzy $\beta$-covering rough set models are constructed under a corresponding fuzzy $\beta$-covering approximation space. However, fuzzy $\beta$-covering is difficult to find directly from the real data. Fortunately, fuzzy information granulation provides a reasonable and effective way to obtain fuzzy $\beta$-coverings from the real data. Since fuzzy information granulation is capable of generating multiple fuzzy $\beta$-coverings, we introduce the notion of multifuzzy $\beta$-covering approximation spaces. Fuzzy $\beta$-covering approximation spaces are a special case of multifuzzy $\beta$-covering approximation spaces. Besides, we employ fuzzy $\beta$-neighborhood operators with reflexivity and symmetry to characterize the similarity between samples. In this article, we first present the definition of multifuzzy $\beta$-covering approximation spaces and investigate some useful properties about fuzzy $\beta$-covering. Second, several information measures are explored in the context of multifuzzy $\beta$-covering approximation spaces. On this basis, a novel heuristic fuzzy $\beta$-covering reduction method with the measure of monotone conditional entropy is proposed. Moreover, a general framework of attribute reduction based on fuzzy $\beta$-covering reduction is also designed. Finally, through the comparative and experimental analyses with other four state-of-the-art attribute reduction methods, the effectiveness and superiority of the proposed method are verified. more...

Practical data often suffer from missing attribute values and lack of class labels. A reasonable machine learning scenario involves obtaining certain values and labels at cost on request. In this article, we propose the cost-sensitive active learning through unified evaluation and dynamic selection (CALS) algorithm to handle the learning task in this new scenario. For data representation, we consider misclassification cost, label query cost, and attribute query cost. For the cost/benefit estimation, we design a unified assessment of attribute values and labels with softmax regression. For the selection of attribute value and label, we propose an optimal acquisition scheme with permutation and greedy strategies. We perform experiments with synthetic, benchmark, and domain datasets. The results of the significance test verify the effectiveness of CALS and its superiority over cost-sensitive active learning and missing data imputation algorithms. more...

• A new imputation approach based on convolutional neural networks is proposed. • A loss function for imputation is designed in the setting of end-to-end learning. • Fuzzy clustering driven data organization is proposed for creating spatial property. The presence of missing data is a challenging issue in processing real-world datasets. It is necessary to improve the data quality by imputing the missing values so that effective learning from data can be achieved. Recently, deep learning has become the most powerful type of machine learning techniques, which can be used for discovering the hidden knowledge that exists in a large dataset to make accurate predictions. In this paper, we propose an imputation method that involves using a convolutional neural network to impute the missing values. The missing value of each instance is imputed essentially by using a trained kernel. The weights of the kernel are determined by learning from the given data that are arranged spatially in the data matrix. The kernel carries out a weighted sum of neighboring elements in an array for imputing the missing values. In addition, in the absence of the true values with which the missing values are expected to be replaced, a loss function is designed without the need to know the true value. Our method is evaluated on UCI datasets in comparison with state-of-the-art methods. The experimental results show that the proposed approach performs closely to or better than other methods. [ABSTRACT FROM AUTHOR] more...

The classification in imbalanced datasets is one of the main problems for machine learning techniques. Support vector machine (SVM) is biased to the majority class samples, and the minority class samples may incorrectly be considered as noise. Therefore, SVM has poor predictive accuracy for imbalanced datasets and generates inaccurate classification models. Existing class imbalance learning (CIL) techniques can make SVM less sensitive to class imbalance, but these methods suffer from issues related to noise and outliers. Moreover, despite the solid theoretical basis and good classification performance, SVM is not appropriate for the classification of large-scale datasets because the training complexity of SVM is closely related to the dataset size. Class imbalance learning (CIL) using Fuzzy adaptive resonance theory (ART) and intuitionistic fuzzy twin SVM (CIL-FART-IFTSVM), which can be applied to address the class imbalance issue in the presence of noise and outliers and large scale datasets, is proposed to overcome these substantial difficulties. In this method, we modify the distribution of the datasets using fuzzy adaptive resonance theory (Fuzzy ART) as a clustering method to overcome the imbalance problem. Then, after data reduction, IFTSVM is utilized to find excellent non-parallel hyperplanes in the generated data points. Finally, a coordinate descent system with shrinking by an active set is applied to reduce the computational complexity. Forty-five imbalanced datasets are considered to validate the performance of the proposed CIL-FART-IFTSVM method. The Friedman test and the bootstrap technique with 95 % confidence intervals are applied to quantify the results statistically. The experimental results indicate that the method proposed in this paper has a better performance compared with other methods, and the training time is significantly better than that of other classifiers for large-scale datasets. [ABSTRACT FROM AUTHOR] more...

This paper provides new insight into the analysis on the relationship between uncertainty and misclassification of a classifier. We formulate the relationship explicitly by taking entropy as a measurement of uncertainty and by analyzing the misclassification rate based on the membership degree difference. Focusing on binary classification problems, this study theoretically and experimentally validates that the misclassification rate will definitely be upgrading with the increase of uncertainty if two conditions are satisfied: (1) the distributions of two classes based on membership degree difference are unimodal, and (2) these two distributions attain peaks when the membership degree difference is less and larger than zero, respectively. This work aims to provide some practical guidelines for improving classifier performance through clearly expressing and understanding the relationship between uncertainty and misclassification of a classifier. [ABSTRACT FROM AUTHOR] more...

In real life, it is a common phenomenon that different misclassification causes different cost. Given a misclassification cost matrix (MCM), cost-sensitive learning is aiming at decreasing the overall misclassification cost rather than simply reducing the misclassification rate. Weighted least squares (WLS) model is acknowledged as an effective way of cost sensitive learning. However, the weights in WLS model are generally unknown and finding these weights is usually difficult. In this paper, we put forward a new approach to learning these weights of WLS model from a given MCM based on a genetic algorithm. A comparative study shows that our proposed approach has an overall cost of misclassification significantly smaller than the existing cost-sensitive learning methods. more...

Fuzzy decision tree (FDT) is an extension of decision tree. Fuzzy classification rules can be extracted by FDT from fuzzy decision tables with fuzzy conditional attributes and fuzzy decision attributes. However, it is very time consuming for fuzzifying conditional attributes, and fuzzification of conditional attributes will inevitably lead to information loss. In order to deal with this problem, based on tolerance rough fuzzy set, this paper proposed an algorithm named TRFDT (Tolerance Rough Fuzzy Decision Tree) and theoretically proved that the proposed algorithm is convergent with a very large probability. TRFDT can directly handle fuzzy decision tables with continuous-valued conditional attributes and fuzzy decision attributes. Accordingly, TRFDT has fast learning speed and good generalization ability, which have been experimentally proved by comparing TRFDT with two state-of-the-art approaches fuzzy ID3 and FDT-YS. [ABSTRACT FROM AUTHOR] more...

Feature selection is viewed as an important preprocessing step for pattern recognition, machine learning, and data mining. Neighborhood is one of the most important concepts in classification learning and can be used to distinguish samples with different decisions. In this paper, a neighborhood discrimination index is proposed to characterize the distinguishing information of a neighborhood relation. It reflects the distinguishing ability of a feature subset. The proposed discrimination index is computed by considering the cardinality of a neighborhood relation rather than neighborhood similarity classes. Variants of the discrimination index, including joint discrimination index, conditional discrimination index, and mutual discrimination index, are introduced to compute the change of distinguishing information caused by the combination of multiple feature subsets. They have the similar properties as Shannon entropy and its variants. A parameter, named neighborhood radius, is introduced in these discrimination measures to address the analysis of real-valued data. Based on the proposed discrimination measures, the significance measure of a candidate feature is defined and a greedy forward algorithm for feature selection is designed. Data sets selected from public data sources are used to compare the proposed algorithm with existing algorithms. The experimental results confirm that the discrimination index-based algorithm yields superior performance compared to other classical algorithms. [ABSTRACT FROM AUTHOR] more...

One of the central problems in object recognition is to develop appropriate representations for the objects in images. The authors present a novel approach for image representation that is based on graphs. In the proposed image graph, each node represents a patch and edges are added between neighbouring nodes. First, class-specific match-set graphs are generated by matching the image graphs that are in the same categories, and the multi-image matching problem is solved by applying a seed-expansion strategy. Then, the matches between the match-set graphs and an image graph are considered to be the object patches in the image. Finally, the features extracted from these patches are used for the image representation. Extensive experiments are conducted to demonstrate that their approach can obtain state-of-the-art results on several challenging datasets. more...

All six papers in this special section engage in different streams but extremely relevant domain vectors of Cognitive Cyber-Physical Systems (CCPS) with artificial intelligence (AI) based solutions in medical informatics. Highlights recent trends in the scientific community and presents emergent technologies, implementations, applications concerning the CPSS. CPSS is witnessing rapid transformation as an interdisciplinary technology that blends physical components and computing devices to enable AI-based solutions. CCPS will be playing a significant role that integrates machine learning/AI techniques and resulting in dramatic improvements for medical informatics and the future of human-augmentation. CPHMS coordinates supervisory medical systems and medical resources everywhere; there is a great scope towards health consciousness and healthy society. Medical Cyber-Physical Systems (MCPS) in healthcare towards critical integration in network of medical devices. MCPS is the next generation computing that is comprised of tightly coupled computational and communication components of medical automation systems such as clinical decision, early detection of health infectious, disease prevention, rapid analysis of health hazards and so on. CCPS and MCPS research would be created new models, new design, and integration models for large scale systems in comprehensive, holistic medical automation systems. With recent enlargements in the big data processing, cognitive data science and AI, it is now possible to create even more realistic digital twins that properly model different operating situations and characteristics to process the medical intelligence systems. more...

With the development of society, although the way that people get information more and more convenient, the information which people get may be incomplete and has a little degree of uncertainty and fuzziness. In real life, the incomplete fuzzy phenomenon of information source exists widely. It is extremely meaningful to fuse multiple fuzzy incomplete information sources effectively. In this study, a new method is presented for information fusion based on information entropy in fuzzy incomplete information system and the effectiveness of the new method is verified by comparing the average fusion method. Then, an illustrative example is delivered to illustrate the effectiveness of the proposed fusion method. Finally, we have also tested the veracity and validity of this method by experiment on a dataset from UCI. The results of this study will be useful for pooling the uncertain data from different information sources and significant for establishing a distinct direction of the fusion method. more...

This paper first proposes a discrete differential evolution algorithm for discounted {0-1} knapsack problems (D{0-1}KP) based on feasible solutions represented by the 0-1 vector. Subsequently, based on two encoding mechanisms of transforming a real vector into an integer vector, two new algorithms for solving D{0-1}KP are given through using integer vectors defined on {0, 1, 2, 3}n to represent feasible solutions of the problem. Finally, the paper conducts a comparative study on the performance between our proposed three discrete differential evolution algorithms and those developed by common genetic algorithms for solving several types of large scale D{0-1}KP problems. The paper confirms the feasibility and effectiveness of designing discrete differential evolution algorithms for D{0-1}KP by encoding conversion approaches. more...

Instance selection is an important preprocessing step in machine learning. By choosing a subset of a data set, it achieves the same performance of a machine learning algorithm as if the whole data set is used, and it enables a machine learning algorithm to be feasible for and to work effectively with large data sets. Based on voting mechanism, this paper proposes a large data sets instance selection algorithm with MapReduce and random weight networks (RWNs). Firstly, the proposed algorithm employs the Map of MapReduce to partition the large data sets into some small subsets, and deploys them to different cloud computing nodes. Secondly, the informative instances are selected in parallel with an instance selection algorithm. Thirdly, the Reduce of MapReduce is used to collect the selected instances from different cloud computing nodes and a selected instance subset is obtained. The previous three processes are repeated p times ( p is a parameter defined by the user), and p instance subsets are obtained. Finally, the voting method is used to select the most informative instances from the p subsets. The random weight network classifier is trained with the selected instance subset, and the testing accuracy is verified on the testing set. The proposed algorithm is experimentally compared with three state-of-the-art approaches which are CNN, ENN and RNN. The experimental results show that the proposed algorithm is effective and efficient. [ABSTRACT FROM AUTHOR] more...

The transmission properties of a focused laser inside anisotropic material are complex due to the birefringent effect, which has remarkable influence on the light distribution and frequency multiplication efficiency of crystals. Meanwhile, it will also affect the laser micromachining precision of birefringent materials with random polarization. In this study, ray tracing and diffraction integral methods were proposed to develop the mathematical model of a laser focused through an isotropic medium into a KDP crystal. Using these models, the focusing properties and 3D light intensity distribution of a focused laser inside a KDP crystal at different orientations were investigated. The research shows that the size and shape of the E-ray focus will distort, and its peak power density decreases rapidly with the decrease of the angle between the optical axis and the crystal surface. Meanwhile, the focal position of the E-ray will also move with the change of optical axis orientation. Based on the simulated results, an approximate 3D light intensity equation of a laser focused into birefringent material was also proposed, which is in good agreement with the theoretical analysis. The related simulated results have an important engineering value for nonlinear optics and laser processing of birefringent materials. more...

Abstract: In this paper, by strict mathematical reasoning, we discover the relationship between the parameters and the reducts in parameterized rough reduction. This relationship, named the nested reduction, shows that the reducts act as a nested structure with the monotonically increasing parameter. We present a systematic theoretical framework that provides some basic principles for constructing the nested structure in parameterized rough reduction. Some specific parameterized rough set models in which the nested reduction can be constructed are pointed out by strict mathematical reasoning. Based on the nested reduction, we design several quick algorithms to find a different reduct when one reduct is already given. Here ‘different’ refers to the reducts obtained on the different parameters. All these algorithms are helpful for quickly finding a proper reduct in the parameterized rough set models. The numerical experiments demonstrate the feasibility and the effectiveness of the nested reduction approach. [Copyright &y& Elsevier] more...

Abstract: Although the traditional rough set theory has been a powerful mathematical tool for modeling incompleteness and vagueness, its performance in dealing with initial fuzzy data is usually poor. This paper makes an attempt to improve its performance by extending the traditional rough set approach to the fuzzy environment. The extension is twofold. One is knowledge representation and the other is knowledge reduction. First, we provide new definitions of fuzzy lower and upper approximations by considering the similarity between the two objects. Second, we extend a number of underlying concepts of knowledge reduction (such as the reduct and core) to the fuzzy environment and use these extensions to propose a heuristic algorithm to learn fuzzy rules from initial fuzzy data. Finally, we provide some numerical experiments to demonstrate the feasibility of the proposed algorithm. One of the main contributions of this paper is that the fundamental relationship between the reducts and core of rough sets is still pertinent after the proposed extension. [Copyright &y& Elsevier] more...

In this paper, an efficient method for solving multiple knapsack problem (MKP) using discrete differential evolution is proposed. Firstly, an integer programming model of MKP suitable for discrete evolutionary algorithm is established. Secondly, a new method for discretizing continuous evolutionary algorithm is proposed based on the combination of transfer function and modulo operation. Therefrom, a new discrete differential evolution algorithm (named TMDDE) is proposed. Thirdly, the algorithm GROA is proposed to eliminate infeasible solutions of MKP. On this basis, a new method for solving MKP using TMDDE is proposed. Finally, the performance of TMDDE using S-shaped, U-shaped, V-shaped, and Taper-shaped transfer functions combined with modulo operation is compared, respectively. It is pointed out that T3-TMDDE which used Taper-shaped transfer function T3 is the best. The comparison results of solving 30 MKP instances show that the performance of T3-TMDDE is better than five advanced evolutionary algorithms. It not only indicates that TMDDE is more competitive for solving MKP, but also demonstrates that the proposed discretization method is an effective method. [ABSTRACT FROM AUTHOR] more...

The quality of an image affects the performance of computer vision applications. The presence of haze often greatly depreciates the visual effect of images. It is a traditional and critical vision challenge to remove haze from a single image. This paper proposes a trainable end-to-end de-hazing connectionist model with a special design. First, feature learning is conducted using hierarchical convolutional layers with nested structures. Cascaded haze-relevant tasks are then sequentially performed via a physics-driven sub-network. In particular, to break the assumption of a homogeneous atmosphere, a branch of the sub-network estimates the scattering factor in the form of a two-dimensional tensor. Finally, a chromatic adaptation layer is proposed for color adjustment, which is often neglected in existing de-hazing methods. In addition, we integrate different training criteria based on the characteristics of the haze-relevant variables in our model. For a fully actionable optimization, an asynchronous learning paradigm is designed for the fusion of different de-hazing tasks, and the joint model is further facilitated by a cyclic restoration. The effectiveness of the proposed de-hazing model was verified via extensive experiments, and most results of our method are remarkable. [ABSTRACT FROM AUTHOR] more...

In this paper, we firstly study two pairs of covering-based fuzzy rough set models and propose the TOPSIS-WAA method based on a covering-based fuzzy rough set. Subsequently, we design a rating scheme based on a multi-criteria decision-making method in a finite fuzzy covering approximation space. The rating scheme is implemented based on the preset subjective ratio and the calculated objective ranking of all alternatives. Furthermore, we use the relevant data of some customers of Industrial and Commercial Bank of China (ICBC) to illustrate the feasibility of our method. At the same time, two test rules are used to verify the validity of the proposed method. Moreover, we compare five different decision-making methods with our method to demonstrate the superiority of our method. Finally, the performance of our method is verified from the perspectives of the best alternative and the optimal ranking. [ABSTRACT FROM AUTHOR] more...

Incremental feature selection can improve learning of accumulated data. We focus on incremental feature selection based on rough sets, which along with their generalizations (e.g., fuzzy rough sets), reduce dimensionality without requiring domain knowledge, such as data distributions. By analyzing the basic concepts of fuzzy rough sets on incremental datasets, we propose incremental mechanisms of information measure. Moreover, we introduce a key instance set containing representative instances to select supplementary features when new instances arrive. As the key instance set is much smaller than the whole dataset, the proposed incremental feature selection mostly suppresses redundant computations. We experimentally compare the proposed method with various non-incremental and two state-of-the-art incremental methods on a variety of datasets. The comparison results demonstrate that the proposed method achieves compact results with reduced computation time, especially on high-dimensional datasets. [ABSTRACT FROM AUTHOR] more...

The knapsack problem with setup (KPS) is a combinatorial optimization problem with important application in the industrial field. In order to solve KPS more quickly and effectively with evolutionary algorithms, a new mathematical model is first established. On the basis of the random algorithm RGSA to generate the potential solution and the repair and optimization algorithm gROA to handle with the infeasible solution, an algorithm framework EA -KPS for solving KPS is given by using evolutionary algorithm. According to EA -KPS, a heuristic algorithm RA-GTOA for solving KPS is proposed by group theory-based optimization algorithm. The comparison of calculation results between RA-GTOA and six representative algorithms for solving 200 KPS benchmark instances shows that RA-GTOA is superior to others in solution accuracy, speed and robustness. This not only shows that RA-GTOA is an efficient algorithm for solving KPS, but also demonstrates that using evolutionary algorithms to solve KPS is an effective method. • A new model of knapsack problem with setup (KPS) is established. • An algorithm for eliminating infeasible solutions of KPS is proposed. • The basic framework of solving KPS is advanced based on evolutionary algorithms. • Algorithm RA-GTOA for KPS is proposed by group theory-based optimization algorithm. • The calculation results show that RA-GTOA is a fast and effective for KPS. [ABSTRACT FROM AUTHOR] more...

Attribute reduction, also known as feature selection, is a common problem by selecting a subset of relevant attributes (e.g. features) to reach efficient learning/mining. Many attribute reduction methods have been proposed however, quite often, these methods are still computationally time-consuming while handling large-scale data. To overcome this shortcoming, we present a novel accelerator based on the positive region, by deleting the learned/discernible instance pairs in the process of attribute reduction, which can avoid redundant computation and accelerate attribute reduction. Our experiments numerically demonstrate that the proposed accelerator can reach drastically faster computation than previous methods, especially on the datasets with a large number of instances. [ABSTRACT FROM AUTHOR] more...

This paper proposes a feature transformation method to improve the performance of clustering and classification, which is named as weight-matrix learning (WML). A feed-forward neural network is particularly designed for WML, which aims to learn the optimal weights by minimizing an objective function similar to cross-entropy, and the training process is finished based on the technique of batch gradient descent or stochastic gradient descent. The proposed feature transformation is linear, which is a non-trivial extension of a previous technique named feature-weight learning (FWL). Essentially, WML can be considered as a learning technique of departing 0.5-similarity, since it can make the samples with similarity larger than 0.5 closer and the samples with similarity lower than 0.5 farther away. From this perspective, WML is identified as an off-center technique with the center of 0.5-similarity. Theoretically and experimentally, it is validated that WML can significantly improve the performance of some clustering algorithms like k -means, and enhance the performance of some classification algorithms like random weight neural network. [ABSTRACT FROM AUTHOR] more...

Uncertainty learning is an important research direction of rough set theory, wherein the most popular one is rough set-based prediction, whose goal is to extract decision rules from decision systems and then assign the corresponding decision labels for new samples in terms of the decision rules. To design efficient prediction algorithms, it is necessary and meaningful to measure the uncertainty of rough set-based prediction, especially the stability and generalization performance. In this paper, we analyze the generalization performance of rough set-based prediction algorithms in terms of algorithmic stability analysis and give the generalization error bounds. Firstly, we propose a general rough set-based prediction algorithm to predict the labels for new samples, and then define a scoring function and the corresponding loss function. Secondly, we define two kinds of algorithmic stability for this prediction algorithm in terms of their loss functions, by which two general generalization error bounds are obtained according to two different kinds of stability: strong stability and pointwise hypothesis stability. The bounds numerically imply the performance of the proposed rough set-based prediction algorithm is related to the number of samples and stability parameter. Thirdly, we adopt the confidence and max confidence, min support algorithms as the specific scoring functions instead of general scoring functions. The results show the prediction performance of the confidence algorithm is related to the number of samples and stability parameter, as well as that of max confidence, min support algorithm is associated with the number of samples and minimum support threshold. Based on these discussions, a general framework of stability and generalization error bounds analysis for rough set-based prediction is established. Finally, several experiments are performed to test the previous conclusions. [ABSTRACT FROM AUTHOR] more...

The artificial bee colony (ABC) algorithm is a powerful population-based metaheuristic for global numerical optimization and has been shown to compete with other swarm-based algorithms. However, ABC suffers from a slow convergence speed. To address this issue, the natural phenomenon in which good individuals always have good genes and thus should have more opportunities to generate offspring is the inspiration for this paper. We propose a ranking-based adaptive ABC algorithm (ARABC). Specifically, in ARABC, food sources are selected by bees to search, and the parent food sources used in the solution search equation are all chosen based on their rankings. The higher a food source is ranked, the more opportunities it will have to be selected. Moreover, the selection probability of the food source is based on the corresponding ranking, which is adaptively adjusted according to the status of the population evolution. To evaluate the performance of ARABC, we compare ARABC with other ABC variants and state-of-the-art differential evolution and particle swarm optimization algorithms based on a number of benchmark functions. The experimental results show that ARABC is significantly better than the algorithms to which it was compared. [ABSTRACT FROM AUTHOR] more...

In the framework of rough sets, incremental algorithms can effectively reduce repetitive computations for dynamic datasets by discovering the update principles of relevant knowledge. Considering the variations in attribute sets, incremental methods based on neighborhood granulation provide fast updates to attribute reduction in continuous-valued information systems. However, most of these methods assume that the sample distribution and neighborhood radius remain fixed with changes in the attribute set, which may mislead the granulation process and affect the model's classification ability. In this paper, a new incremental reduction method based on granular balls and attribute grouping is proposed for dynamic information systems with multiple attribute additions. Different neighborhood radii are adaptively determined when the attribute set changes, and the number of neighborhood granules can also be effectively reduced. Furthermore, the attributes are grouped based on the k-means algorithm, and only attributes from different groups or with small relevance to those in the current reduction set are considered to be incorporated as a reduction attribute, thus reducing the computation time and simultaneously preserving informative attributes. Linear and nonlinear correlation coefficients are used to measure attribute relevance based on which three corresponding incremental reduction algorithms are developed. Finally, extensive experimental results on 12 benchmark datasets are shown to compare the proposed method with non-incremental and typical incremental methods in terms of time cost, classification accuracy, the size of attribute reduction, and coverage of the reduction. [ABSTRACT FROM AUTHOR] more...

Various learning problems can be represented as certain canonical forms of orthogonal matrix nearness problems under the unitarily invariant norm. Since varying unitarily invariant norms favor different structured learning of input data, it is crucial to construct a unified scheme to learn encouraging distributions via any unitarily invariant norm. In this paper, we find that the orthogonal matrix nearness problem can be generalized to a common architecture of the unitarily invariant norm minimization, thus further exploit iterative and closed-form solutions. Firstly, we start with several special circumstances of orthogonal matrix nearness problems, where the unitarily invariant norm resorts to Frobenius norm. Secondly, a general scheme for orthogonal matrix nearness problems under any unitarily invariant norm is addressed and the corresponding iterative algorithm is proposed. Thirdly, feature representation problems are formulated as certain forms of orthogonal matrix nearness, indicating a joint framework for learning low-dimensional features with closed-form solutions. Finally, comprehensive experiments on real-world data sets demonstrate the effectiveness of the proposed method against compared state-of-the-art feature representation approaches. In particular, the proposed method gains about 15% improvements of clustering accuracy on BASEHOCK, CNAE and RCV1 data sets than benchmarks, and achieves at least 3% improvements on other data sets. [ABSTRACT FROM AUTHOR] more...

Multi-label active learning (MLAL) reduces the cost of manual annotation for multi-label problems by selecting high-quality unlabeled data. Existing MLAL methods usually perform a one-way selection by considering the examples' informativeness, representativeness, or diversity. These methods acknowledge only the importance of the examples to the labels, but not vice versa. Due to the imbalanced nature of multi-label data, the selected dataset might also be highly imbalanced, causing negative effects on the learning performance. In this paper, we treat the selection of example-label pairs in MLAL as a two-way matching problem instead of a one-way selection problem. First, the label's preference for example, defined as the informativeness of the example regarding the label, as well as the example's preference for the label, which is defined as the probability of the example belonging to a positive class, are both considered. Then, a simple and effective stable matching (STM) model is adopted to realize the two-way selection. In addition, to provide reasonable candidates for the STM model, the roulette algorithm is utilized to allocate the annotation number for sub-classifiers. Comprehensive experiments demonstrate the competitiveness of the proposed approach and its effectiveness in selecting a relatively balanced dataset. [ABSTRACT FROM AUTHOR] more...

Abstract: Active learning approach has been integrated with support vector machine or other machine-learning techniques in many areas. However, the challenge is: Unlabeled instances are often abundant or easy to obtain, but their labels are expensive and time-consuming to get in general. In spite of this, most existing methods cannot guarantee the usefulness of each query in learning a new classifier. In this paper, we propose a new active learning approach of selecting the most informative query for annotation. Unlabeled instance, which is nearest to the support vector machine’s hyperplane learnt from both the unlabeled instance itself and all labeled instances, is selected as the query for annotation. Merits of these queries in learning a new optimal hyperplane have been assured before they are annotated and put into the training set. Experimental results on several UCI datasets have shown the efficiency of our approach. [Copyright &y& Elsevier] more...

Fuzzy rough sets have strong ability in dealing with data uncertainty, which have been widely applied in attribute reduction problems. Fuzzy relations are the cornerstones of fuzzy rough sets, and kernel-induced relations can simultaneously reflect the similarities of objects in both original space and high-dimensional Hilbert space. However, existing distance-based fuzzy relations (even kernel-induced) cannot truly reflect the correlations of objects under complex data distributions. To obtain better expressing ability for data, we propose a general optimization framework to get the memberships of objects with respect to categories. Considering various complex data distributions, the concepts of intra-class aggregation degree based on membership and inter-class dispersion degree based on nonmembership are proposed, and a separability measure is constructed to evaluate the importance of attributes to decisions. Correspondingly, a new attribute reduction algorithm is proposed, with a forward evaluation L -step strategy to skip local fluctuations in classification accuracy. Finally, we conduct a series of experiments to verify the effectiveness of the proposed algorithm. From the experiments on UCI, ELVIRA and face recognition datasets, it can be seen that the attribute subset obtained by the proposed algorithm not only has high classification accuracy and computational efficiency, but also has fewer attributes. [ABSTRACT FROM AUTHOR] more...

In rough set philosophy, each set of data can be seen as a fuzzy decision table. Since a decision table dynamically increases with time and space, these decision tables are integrated into a new one called fused decision table. In this paper, we focus on designing an incremental feature selection method on fused decision table by optimizing the space constraint of storing discernibility matrix. Here discernibility matrix is a known way of discernibility information measure in rough set theory. This paper applies the quasi/pseudo value of discernibility matrix rather than the true value of discernibility matrix to design an incremental mechanism. Unlike those discernibility matrix based non-incremental algorithms, the improved algorithm needs not save the whole discernibility matrix in main memory, which is desirable for the large data sets. More importantly, with the increment of decision tables, the discernibility matrix-based feature selection algorithm could constrain the computational cost by applying efficient information updating techniques—quasi/pseudo approximation operators. Finally, our experiments reveal that the proposed algorithm needs less computational cost, especially less occupied space, on the condition that the accuracy is limitedly lost. [ABSTRACT FROM AUTHOR] more...

Stochastic Configure Networks (SCNs) are an incremental variant of randomly weighted neural networks of which one of the key highlights lies in the constraints while adding hidden layer nodes. Recent studies show two weaknesses of SCNs. One is the redundancy of added hidden layer nodes while the other is the lack of interpretability to the designed constraints. Motivated by overcoming these weaknesses, this paper proposes a new model named Evolving SCN to increase the interpretability of the design for constraints from the viewpoint of evolution by a sampling mechanism and to promote the model compactness by optimizing random weights within the space of constraint parameters. Surprisingly, although an evolving method is used in our model, the effectiveness and efficiency are significantly improved with respect to running time and prediction accuracy in comparison with the existing versions of SCNs. This work makes a first attempt to enhance the interpretability of incrementally adding nodes and simultaneously reduce the redundancy of hidden nodes in SCNs, which brings some new insights to understand the model compactness from the view of Occam's razor and further to the nature of incremental learning. • An accessible interpretation of the designed constraints in SCN is summarized, extending the fundamental theory. • A new evolution mechanism is proposed to optimize random weights, addressing hidden node redundancy in SCN, i.e., E-SCN. • Real-world dataset experiments reveal E-SCN's improved compactness, training efficiency, and generalization ability. [ABSTRACT FROM AUTHOR] more...

In formal concept analysis, many reduction methods have recently been proposed for formal decision contexts, and each of them was to reduce formal decision contexts with a particular purpose. However, little attention has been paid to the comparison of their differences from various aspects. In fact, this problem is very important because it can provide evidence to select an appropriate reduction method for a given specific case. To address this problem, our study mainly focuses on clarifying the relationship among the existing reduction methods in formal decision contexts. Firstly, we give a rule-based review of the existing reduction methods, revealing the type of rules that each of them can preserve. Secondly, we analyze the relationship among the consistencies introduced by the existing reduction methods. More specifically, Wei's first consistency (see [39] ) is stronger than others, while her second one is weaker than the remainder except Wu's consistency (see [43] ). Finally, we make a comparison of the existing reductions, concluding that Li's reduction (see [14] ) maintaining the non-redundant decision rules of a formal decision context is coarser than others. The results obtained in this paper are beneficial for users to select an appropriate reduction method for meeting their requirements. [ABSTRACT FROM AUTHOR] more...

The error-bounded Piecewise Linear Approximation (PLA) is to approximate the stream data by lines such that the approximation error at each point does not exceed a pre-defined error. In this paper, we focus on the version of PLA problem that generates connected lines in the segmentation for smooth approximation. We provide a new linear-time algorithm for the problem that outperform two of the existing methods with less number of connected segments. Our extensive experiments, on both real and synthetic data sets, indicate that our proposed algorithms are practically efficient. [ABSTRACT FROM AUTHOR] more...