Search

This paper develops a weak-label-based global and local multi-view multi-label learning with three-way clustering (WL-GLMVML-ATC) to solve multi-view multi-label data sets and exploit more authentic global and local label correlations of both the whole data set and each view simultaneously. Different from the traditional learning methods, WL-GLMVML-ATC pays more attention to the solutions of weak-label cases and uncertain relationships of clusters with the usage of Universum and active three-way clustering. According to Universum notion, even though the size of labeled instances is much more smaller than the unlabeled ones, the useful sample information can still be enhanced. Through the active three-way clustering strategy, the belongingness of instances to a cluster depend on the probabilities of uncertain instances belonging to core regions. This strategy brings a more authentic local label correlation since many traditional methods suppose that instances and the corresponding clusters always exhibit certain relationships such as belong-to definitely and not belong-to definitely. This hypothesis is not ubiquitous in real-world applications. According to the experiments, we can see WL-GLMVML-ATC (1) achieves a better performance, be superior to the classical multi-view learning methods and multi-label learning methods in statistical, advances the development of these learning methods in final; (2) won’t add too much running time; (3) has a good convergence and ability to process multi-view multi-label data sets.

The tolerance rough set is developed as one of the outstanding extensions of the Pawlak's rough set model under incomplete information, and the limited tolerance relation is developed to overcome the problem that objects leniently satisfy the tolerance relation. However, the classification based on the limited tolerance relationship cannot reflect the matching degree of uncertain information of objects. In this article, we explore the influence of null values in an incomplete system, and propose the constrained tolerance relation based on the matching degree of uncertain information of objects. The proposed rough set based on the constrained tolerance relation can provide a more detailed structure of an object class through threshold. Proofs and example analyses further show the rationality and superiority of the proposed model.

Machine reading comprehension is a very challenging task, which aims to determine the answer span based on the given context and question. The newly developed pre-training language model has achieved a series of successes in various natural language understanding tasks with its powerful contextual representation ability. However, these pre-training language models generally lack the downstream processing structure for specific tasks, which limits further performance improvement. In order to solve this problem and deepen the model’s understanding of the question and context, this paper proposes S&I Reader. On the basis of the pre-training model, skimming, intensive reading, and gated mechanism modules are added to simulate the behavior of humans reading text and filtering information. Based on the idea of granular computing, a multi-granularity module for computing context granularity and sequence granularity is added to the model to simulate the behavior of human beings to understand the text from words to sentences, from parts to the whole. Compared with the previous machine reading comprehension model, our model structure is novel. The skimming module and multi-granularity module proposed in this paper are used to solve the problem that the previous model ignores the key information of the text and cannot understand the text with multi granularity. Experiments show that the model proposed in this paper is effective for both Chinese and English datasets. It can better understand the question and context and give a more accurate answer. The performance has made new progress on the basis of the baseline model.

The theory of three-way decision plays an important role in decision making and knowledge reasoning. However, little attention has been paid to the problem of learning from partially labeled data with three-way decision. In this paper, we propose a three-way co-decision model for partially labeled data. More specifically, the problem of attribute reduction for partially labeled data is first investigated, and two semi-supervised attribute reduction algorithms based on novel confidence discernibility matrix are proposed. Then, a three-way co-decision model is introduced to classify unlabeled data into useful, useless, and uncertain data, and the model is iteratively retrained on the carefully selected useful data to improve its performance. Moreover, we theoretically analyze the effectiveness of the proposed model. The experimental results conducted on UCI data sets demonstrate that the proposed model is promising, and even compares favourably with the single supervised classifier trained on all training data with true labels.

Hierarchical classification (HC) is effective when categories are organized hierarchically. However, the blocking problem makes the effect of hierarchical classification greatly reduced. Blocking means that samples are easily getting misclassified in high-level classifiers so that the samples are blocked at the high-level of the hierarchy. This issue is caused by the inconsistency between the artificially defined hierarchy and the actual hierarchy of the raw data. Another issue is that it is flippant to strictly process data following the hierarchy. Therefore, special treatment is required for some uncertain data. To address the first issue, we learn category relationships and modify the hierarchy. To address the second issue, we introduce three-way decisions (3WD) to targetedly deal with the ambiguous data. We extend original studies and propose two HC models based on 3WD, collectively referred to as TriHC, for carefully modifying the hierarchy to alleviate the blocking problem. The proposed TriHC model learns new category hierarchies by the following three steps: (1) mining category relations; (2) modifying category hierarchies according to the latent category relations; and (3) using 3WD to divide observed objects into three regions: positive region, boundary region, and negative region, and making decisions based on different strategies. Specifically, based on different category relation mining methods, there are two versions of TriHC, cross-level blocking priori knowledge based TriHC (CLPK-TriHC) and expert classifier based TriHC (EC-TriHC). The CLPK-TriHC model defines a cross-level blocking distribution matrix to mine the category relations between the higher and lower levels. To better exploit category hierarchical relations, the EC-TriHC model builds expert classifiers using topic model to learn latent category topics. Experimental results validate that the proposed methods can simultaneously reduce the blocking and improve the classification accuracy.

Person Re-identification (ReID) aims to retrieve the pedestrian with the same identity across different views. Existing studies mainly focus on improving accuracy, while ignoring their efficiency. Recently, several hash based methods have been proposed. Despite their improvement in efficiency, there still exists an unacceptable gap in accuracy between these methods and real-valued ones. Besides, few attempts have been made to simultaneously explicitly reduce redundancy and improve discrimination of hash codes, especially for short ones. Integrating Mutual learning may be a possible solution to reach this goal. However, it fails to utilize the complementary effect of teacher and student models. Additionally, it will degrade the performance of teacher models by treating two models equally. To address these issues, we propose a salience-guided iterative asymmetric mutual hashing (SIAMH) to achieve high-quality hash code generation and fast feature extraction. Specifically, a salience-guided self-distillation branch (SSB) is proposed to enable SIAMH to generate hash codes based on salience regions, thus explicitly reducing the redundancy between codes. Moreover, a novel iterative asymmetric mutual training strategy (IAMT) is proposed to alleviate drawbacks of common mutual learning, which can continuously refine the discriminative regions for SSB and extract regularized dark knowledge for two models as well. Extensive experiment results on five widely used datasets demonstrate the superiority of the proposed method in efficiency and accuracy when compared with existing state-of-the-art hashing and real-valued approaches. The code is released at https://github.com/Vill-Lab/SIAMH .

Neighborhood Covering (NC) is the union of homogeneous neighborhoods and provides a set-level approximation of data distribution. Because of the nonparametric property and the robustness to complex data, neighborhood covering has been widely used for data classification . Most existing methods directly classify data samples according to the nearest neighborhoods. However, the certain classification methods strictly classify the uncertain data and may lead to serious classification mistakes. To tackle this problem, we extend traditional neighborhood coverings to fuzzy ones and thereby propose a Three-Way Classification method with Fuzzy Neighborhood Covering (3WC-FNC). Fuzzy neighborhood covering consists of membership functions and forms an approximate distribution of neighborhood belongingness. Based on the soft partition induced by the memberships of fuzzy neighborhood coverings of different classes, data samples are classified into Positive (certainly belonging to a class), Negative (certainly beyond classes) and Uncertain cases. Experiments verify that the proposed three-way classification method is effective to handle the uncertain data and in the meantime reduce the classification risk.

In order to process multi-view multi-label data sets, we propose global and local multi-view multi-label learning (GLMVML). This method can exploit global and local label correlations of both the whole data set and each view simultaneously. What’s more, GLMVML introduces a consensus multi-view representation which encodes the complementary information from different views. Related experiments on three multi-view data sets, fourteen multi-label data sets, and one multi-view multi-label data set have validated that (1) GLMVML has a better average AUC and precision and it is superior to the classical multi-view learning methods and multi-label learning methods in statistical; (2) the running time of GLMVML won’t add too much; (3) GLMVML has a good convergence and ability to process multi-view multi-label data sets; (4) since the model of GLMVML consists of both the global label correlations and local label correlations, so parameter values should be moderate rather than too large or too small.

Semi-supervised multi-view learning machine is developed to process the corresponding semi-supervised multi-view data sets which consist of labeled and unlabeled instances. But in real-world applications, for a multi-view data set, only few instances are labeled with the limitation of manpower and cost. As a result, few prior knowledge which is necessary for the designing of a learning machine is provided. Moreover, in practice, different views and features play diverse discriminant roles while traditional learning machines treat these roles equally and assign the same weight just for convenience. In order to solve these problems, we introduce Universum learning to obtain more prior knowledge and assign different weights for views and features to reflect their diverse discriminant roles. The proposed learning machine is named as weight-and-Universum-based semi-supervised multi-view learning machine (WUSM). In WUSM, we first obtain weights of views and features. Then, we construct Universum set to obtain more prior knowledge on the basis of these weights. Different from traditional construction ways, the used construction way makes full use of the information of all labeled and unlabeled instances rather than only a pair of positive and negative training instances. Finally, we design the machine with the usage of the Universum set along with original data set. Our contributions are given as follows. (1) With the usage of all (labeled, unlabeled) instances of the data set, the Universum set provides more useful prior knowledge. (2) WUSM considers the diversities of views and features. (3) WUSM advances the development of semi-supervised multi-view learning machines. Experiments on bipartite ranking, feature selection, dimensionality reduction, classification, clustering, etc. validate the advantages of WUSM and draw a conclusion that with the introduction of Universum learning, view weights, and feature weights, the performance of a semi-supervised multi-view learning machine is boosted.

Data and rules power expert systems and intelligent systems. Rules, as a form of knowledge representation, can be acquired by experts or learned from data. The accuracy and precision of knowledge largely determines the success of the systems, which awakens the concern for causality. The ability to elicit cause–effect rules directly from data is key and difficult to any expert systems and intelligent systems. Rough set theory has succeeded in automatically transforming data into knowledge, where data are often presented as an attribute-value table. However, the existing tools in this theory are currently incapable of interpreting counterfactuals and interventions involved in causal analysis. This paper offers an attempt to characterize the cause–effect relationships between attributes in attribute-value tables with intent to overcome existing limitations. First, we establish the main conditions that attributes need to satisfy in order to estimate the causal effects between them, by employing the back-door criterion and the adjustment formula for a directed acyclic graph. In particular, based on the notion of lower approximation, we extend the back-door criterion to an original data table without any graphical structures. We then identify the effects of the interventions and the counterfactual interpretation of causation between attributes in such tables. Through illustrative studies completed for some attribute-value tables, we show the procedure for identifying the causation between attributes and examine whether the dependency of the attributes can describe causality between them.

Traditional supervised multi-view learning machines aim to process multi-view data sets which consist of labeled instances from multiple views. While they cannot deal with semi-supervised data sets whose training instances consist of both labeled and unlabeled ones. Moreover, with the limitation of storage and process ability, some learning machines cannot process large-scale data sets. Furthermore, some instances maybe have missing features or views and traditional multi-view learning machines have no ability to process the data sets with variable features and views. Thus, this paper develops a semi-supervised one-pass multi-view learning with variable features and views (SOMVFV) so as to process the large-scale semi-supervised data sets with variable features and views. Related experiments on some supervised, semi-supervised, large-scale, and small-scale data sets validate the effectiveness of our proposed SOMVFV and we can get the following conclusions, (1) SOMVFV can process multiple kinds of special data sets; (2) compared with most learning machines used in our experiments, the better performance of SOMVFV is significant; (3) compared with missing views, missing features has a greater influence on the classification accuracy.

Deep neural network models have achieved remarkable results in sentiment classification. Traditional feature-based methods perform slightly worse than deep learning methods in terms of classification accuracy , but they have their own advantages in interpretability and time complexity. To the best of our knowledge, few works study the ensemble of deep learning methods and traditional feature-based methods. Inspired by the methodology of three-way decisions, we proposed a three-way enhanced convolutional neural network model named 3W-CNN. 3W-CNN can be seen as an ensemble method which uses the enhance model to optimize convolutional neural networks (CNN). The enhance model is selected according to the classification accuracy and the difference in classification results compared to CNN. Support vector machine with naive bayes features (NB-SVM) is selected as the enhance model after comparing with several baseline models . However, the performance of NB-SVM is worse than CNN on most of benchmark datasets. To address this issue, we construct a component named confidence divider and design a confidence function to distinguish the classification quality of CNN. NB-SVM is further utilized to reclassify the predictions with weak confidence. The experimental results validated the effectiveness of 3W-CNN and showed three-way decisions could further improve the accuracy of sentiment classification.

Multi-view clustering aims to group multi-view samples into different clusters based on the similarity. Since side information can describe the relation between samples, for example, must-links and cannot-links, thus multi-view clustering with the consideration about side information along with samples can get more feasible clustering results. As a recent developed multi-view clustering approach, multi-view matrix completion (MVMC) constructs similarity matrix for each view and casts clustering into a matrix completion problem. Different from traditional multi-view clustering approaches, MVMC enforces the consistency of clustering results on different views as constraints for alternative optimization and the global optimal solution can be obtained. Although related experiments show that MVMC exhibits impressive performance, it still neglects the possibility of a sample belonging to a cluster. In this paper, we consider the possibility on the base of entropy and develop an entropy-based multi-view matrix completion for clustering with side information (EMVMC). Experiments on multi-view datasets Course, Citeseer, Cora, WebKB, NewsGroup, and Reuters validate the effectiveness of EMVMC.

Uncertainty is a fundamental and unavoidable feature in daily life, which is the same for a single classifier. Thus, combining the predictions of many different classifiers is a very successful way to reduce the uncertainty. In this paper, we present a Correcting Reliability Level (CRL) supervised three-way decision (3WD) cascade model to implement image classification tasks. Our model simulates the human decision process by using 3WD to judge “certainty” or “uncertainty” of the classification result. When judged as “uncertainty”, CRL will supervise the 3WD and learn more information to make the final prediction. In addition, we introduce two Class Grouping methods to mining the relation between labels, which help us to train several expert ConvNets for different types of images. Experimental results show that our model can effectively reduce the classification error rate compared with the base classifier.

Label ambiguity and data complexity are widely recognized as major challenges in multi-label classification. Existing studies strive to find approximate representations concerning label semantics, however, most of them are predefined, neglecting the personality of instance-label pair. To circumvent this drawback, this paper proposes a three-way selective ensemble (TSEN) model. In this model, three-way decisions is responsible for minimizing uncertainty, whereas ensemble learning is in charge of optimizing label associations. Both label ambiguity and data complexity are firstly reduced, which is realized by a modified probabilistic rough set. For reductions with shared attributes, we further promote the prediction performance by an ensemble strategy. The components in base classifiers are label-specific, and the voting results of instance-based level are utilized for tri-partition. Positive and negative decisions are determined directly, whereas the deferment region is determined by label-specific reduction. Empirical studies on a collection of benchmarks demonstrate that TSEN achieves competitive performance against state-of-the-art multi-label classification algorithms.

Attribute reduction in rough set theory serves as a fundamental topic for information processing, and its basis is usually a decision table (D-Table). D-Table attribute reduction concerns three hierarchical types, and only classification-based reduction is related to information-theoretic representation. Aiming at inducing comprehensive D-Table attribute reduction with hierarchies and information, this paper concretely constructs a D-Table’s three-layer granular structures and three-way informational measures via granular computing and Bayes’ theorem. With regard to the D-Table, the micro-bottom, meso-middle, and macro-top are hierarchically organized according to the formal structure and systematic granularity. Then, different layers produce different three-way informational measures by developing Bayes’ theorem. Thus, three-way weighted entropies originate from three-way probabilities at the micro-bottom and further evolve from the meso-middle to the macro-top, and their granulation monotonicity and evolution systematicness are acquired. Furthermore, three-way informational measures are analyzed by three-layer granular structures to achieve their hierarchical evolution, superiority, and algorithms. Finally, structural and informational results are effectively illustrated by a D-Table example. This study establishes D-Table’s hierarchical structures to reveal constructional mechanisms and systematic relationships of informational measures. The obtained results underlie the D-Table’s hierarchical, systematic, and informational attribute reduction, and they also enrich the three-way decisions theory.

In various real-world situations, there are actually a large number of dynamic covering information systems, and non-incremental learning technique is time consuming for updating approximations of sets in dynamic covering information systems. In this paper, we investigate incremental mechanisms of updating the second and sixth lower and upper approximations of sets in dynamic covering information systems with variations of attributes. Especially, we design effective algorithms for calculating the second and sixth lower and upper approximations of sets in dynamic covering information systems. The experimental results indicate that incremental algorithms outperform non-incremental algorithms in the presence of dynamic variation of attributes. Finally, we explore several examples to illustrate that the proposed approaches are feasible to perform knowledge reduction of dynamic covering information systems.

Three-way decisions are a fundamental methodology with extensive applications, while attribute reducts play an important role in data analyses. The combination of both topics has theoretical significance and applicable prospects, but rarely gains direct research at present. In this paper, three-way decisions are introduced into attribute reducts and thus three-way attribute reducts are systematically investigated. Firstly, classical qualitative reducts are reviewed by the dependency degree. Then, the dependency degree implements approximation analyses to be improved to a controllable measure: the relative dependency degree, which is monotonic to relatively measure the attribute dependency. Given an approximate bar, the relative dependency degree defines the applicable quantitative reducts, which approach, expand, and weaken the classical qualitative reducts. This type of quantitative reducts is actually the positive quantitative reducts for three-way reducts. Thus, three-way quantitative reducts are established by the relative dependency degree and dual thresholds. The positive, boundary, and negative quantitative reducts divide the power set of the condition attribute set and thus gain acceptance, noncommitment, and rejection decisions, respectively; they exhibit the potential derivation from the higher level to the lower level. Furthermore, three-way qualitative reducts are established by degeneration to implement three-way decisions, and three-way quantitative and qualitative reducts exhibit the approximation, expansion, and strength; by virtue of superiority analyses, three-way reducts improve the latent two-way reducts with only acceptance and rejection decisions. Finally, three-way reducts are practically illustrated by observing an example of decision tables. By developing the relative dependency degree with controllability, three-way reducts implement both a quantitative generalization for qualitative reducts and a structural completion for attribute reducts. The relevant study provides a new insight into both three-way decisions and attribute reducts.

Stream computing paradigm, with the characteristics of real-time arrival and departure, has been admitted as a major computing paradigm in big data. Relevant theories are flourishing recently with the surge development of stream computing platforms such as Storm, Kafka and Spark. Rough set theory is an effective tool to extract knowledge with imperfect information, however, related discussions on synchronous immigration and emigration of objects have not been investigated. In this paper, stream computing learning method is proposed on the basis of existing incremental learning studies. This method aims at solving challenges resulted from simultaneous addition and deletion of objects. Based on novel learning method, a stream computing algorithm called single-object stream-computing-based three-way decisions (SS3WD) is developed. In this algorithm, the probabilistic rough set model is applied to approximate the dynamic variation of concepts. Three-way regions can be determined without multiple scans of existing information granular. Extensive experiments not only demonstrate better efficiency and robustness of SS3WD in the presence of objects streaming variation, but also illustrate that stream computing learning method is an effective computing strategy for big data.

We granulate the label space into information granules to exploit label dependency.We present a multi-label maximal correlation minimal redundancy criterion.The proposed method can select compact and specific feature subsets.The proposed method can significantly improve the algorithm performance. Like the traditional machine learning, the multi-label learning is faced with the curse of dimensionality. Some feature selection algorithms have been proposed for multi-label learning, which either convert the multi-label feature selection problem into numerous single-label feature selection problems, or directly select features from the multi-label data set. However, the former omit the label dependency, or produce too many new labels leading to learning with significant difficulties; the latter, taking the global label dependency into consideration, usually select a few redundant or irrelevant features, because actually not all labels depend on each other, which may confuse the algorithm and degrade its classification performance. To select a more relevant and compact feature subset as well as explore the label dependency, a granular feature selection method for multi-label learning is proposed with a maximal correlation minimal redundancy criterion based on mutual information. The maximal correlation minimal redundancy criterion makes sure that the selected feature subset contains the most class-discriminative information, while in the meantime exhibits the least intra-redundancy. Granulation can help explore the label dependency. We study the relation of the label granularity and the performance on four data sets, and compare the proposed method with other three multi-label feature selection methods. The experimental results demonstrate that the proposed method can select compact and specific feature subsets, improve the classification performance and performs better than other three methods on the widely-used multi-label learning evaluation criteria.

Keyphrase extraction has been a popular research topic in the field of natural language processing in recent years. But how to extract keyphrases precisely and effectively is still a challenge. The mainstream methods are supervised learning methods and graph-based methods. Generally, the effects of supervised methods are better than unsupervised methods. However, there are many problems in supervised methods such as the difficulty in obtaining training data, the cost of labeling and the limitation of the classification function trained by training data. In recent years, the development of the graph-based method has made great progress and its performance of extraction is getting closer and closer to the supervised method, so the graph-based method of keyphrase extraction has got a wide concern from researchers. In this paper, we propose a new model that applies the three-way decision theory to graph-based keyphrase extraction model. In our model, we propose algorithms dividing the set of candidate phrases into the positive domain, the boundary domain and the negative domain depending on graph-based attributes, and combining candidate phrases in the positive domain and the boundary domain qualified by graph-based attributes and non- graph-based attributes to get keyphrases. Experimental results show that our model can effectively improve the extraction precision compared with baseline methods.

In real-world multi-view multi-label clustering and some classification tasks, instances and the corresponding clusters has at least three kinds of relationships, belong-to definitely, not belong-to definitely, and uncertain. Some learning machines consider only two of them, for example, belong-to definitely and not belong-to definitely. Moreover, three-way decision-based clustering (TDC) strategy is a good method to make the belongingness of instances to a cluster depend on the probabilities of uncertain instances belonging to core regions. Thus in our work, we take the notion of classical multi-view multi-label learning machines as the basic and introduce TDC so as to develop a multi-view and multi-label method with three-way decision-based clustering (MVML-TDC) and consider the relationships between instances and clusters. Experimental results validate that MVML-TDC achieves a better average performance and an acceptable running time.

In multi-view learning field, preserving data privacy is an important topic and a good solution is rank-consistency-based multi-view learning (RANC). RANC exploits view relationship and preserves data privacy simultaneously and related experiments also validate that RANC improves the individual view-specific learners with the usage of information from other views and parts of features. While performance of RANC is still limited by the insufficient of prior knowledge. Thus we introduce Universum learning into RANC to create additional unlabeled instances which provide more useful prior knowledge. The developed RANC with Universum learning is abbreviated to RANCU. Related experiments on some multi-view data sets have validated the performance of our RANCU theoretically and empirically.

In recent years, deep learning has developed rapidly and is widely used in various fields, such as computer vision, speech recognition, and natural language processing. For end-to-end person re-identification, most deep learning methods rely on large-scale datasets. Relatively few methods work with small-scale datasets. Insufficient training samples will affect neural network accuracy significantly. This problem limits the practical application of person re-identification. For small-scale person re-identification, the uncertainty of person representation and the overfitting problem associated with deep learning remain to be solved. Quantifying the uncertainty is difficult owing to complex network structures and the large number of hyperparameters. In this study, we consider the uncertainty of pedestrian representation for small-scale person re-identification. To reduce the impact of uncertain person representations, we transform parameters into distributions and conduct multiple sampling by using multilevel dropout in a testing process. We design an improved Monte Carlo strategy that considers both the average distance and shortest distance for matching and ranking. When compared with state-of-the-art methods, the proposed method significantly improve accuracy on two small-scale person re-identification datasets and is robust on four large-scale datasets.

Multi-view multi-label learning has attracted the attention of many scholars and widely used in multiple fields. While in real-world applications, due to the lack of manpower and equipment failure, the data sets to be processed maybe loss some labels or views. Moreover, most multi-view multi-label learning methods neglect the global and local label correlations of both the whole data set and each view and the complementary information coming from different views sometimes. Furthermore, some methods ignore such a phenomenon that each class label might be determined by some specific features of its own. In order to improve the performance of such methods, in this paper, we develop an improved multi-view multi-label learning with incomplete views and labels (IMVL-IV). In framework of IMVL-IV, the usage of label-specific features makes the decision of label be determined by some specific features rather than all features so that we can pay more attention to portion specific features and save time; the introduction of label correlation matrix offsets the defect of missing labels; the adoption of low-rank assumption matrix restores missing views; global and local label correlations are taken into consideration with clustering technology; a consensus multi-view representation is put to use to encode the complementary information from different views. Different from traditional learning methods, this is the first attempt to design a multi-view multi-label learning method with incomplete views and labels by the learning of label-specific features, label correlation matrix, low-rank assumption matrix, global and local label correlations, and complementary information. Experimental results validate that IMVL-IV achieves a better performance and it is superior to the classical multi-view learning methods and multi-label learning methods.

Knowledge-coarsening is investigated to describe attribute deletion.Granule-merging and its region-distribution are used to gain region-change functions.Region-change with certainty/monotonicity is analyzed in qualitative Pawlak-Model.Region-change with uncertainty/drifting is analyzed in quantitative DTRS-Model. Attribute reduction is a fundamental research theme in rough sets and granular computing (GrC). Its scientific construction originally depends on the region-change law. At present, only region-change non-monotonicity/monotonicity is mined in the quantitative/qualitative model. The in-depth region-change truth and its GrC mechanism have significance, especially for follow-up attribute reduction. This paper commences probing region-change essence, mainly from a novel uncertainty/certainty viewpoint. Concretely, we make comparative region-change analyses based on GrC, by resorting to the qualitative Pawlak-Model and quantitative DTRS-Model (the decision-theoretic rough set model). (1) Knowledge-coarsening is investigated to describe attribute deletion. (2) Granule-merging and its region-distribution are studied to probe region-change functions. (3) Region-change is analyzed in Pawlak-Model to mine qualitative region-change certainty and its relevant properties. (4) Region-change is analyzed in DTRS-Model to mine quantitative region-change uncertainty and its relevant properties. (5) Comparative region-change analyses are summarized, and further experiment verification is provided. Knowledge-coarsening and granule-merging establish GrC mechanisms for extensive region-change analyses. Quantitative/qualitative region-change uncertainty/certainty and relevant principles are discovered via DTRS-Model/Pawlak-Model. By virtue of the GrC technology and comparative strategy, this study reveals region-change uncertainty/certainty to deepen region-change non-monotonicity/monotonicity; furthermore, it underlies attribute reduction, especially with regard to quantitative models.

IP-Accuracy is mined by systematic double-quantitative fusion of causality measures.IP-Accuracy GrC integration is constructed to gain benign granulation monotonicity.IP-Accuracy attribute reduction is studied to establish a hierarchical reduct system. Uncertainty measure mining and applications are fundamental, and it is possible for double-quantitative fusion to acquire benign measures via heterogeneity and complementarity. This paper investigates the double-quantitative fusion of relative accuracy and absolute importance to provide systematic measure mining, benign integration construction, and hierarchical attribute reduction. (1) First, three-way probabilities and measures are analyzed. Thus, the accuracy and importance are systematically extracted, and both are further fused into importance-accuracy (IP-Accuracy), a synthetic causality measure. (2) By sum integration, IP-Accuracy gains a bottom-top granulation construction and granular hierarchical structure. IP-Accuracy holds benign granulation monotonicity at both the knowledge concept and classification levels. (3) IP-Accuracy attribute reduction is explored based on decision tables. A hierarchical reduct system is thereby established, including qualitative/quantitative reducts, tolerant/approximate reducts, reduct hierarchies, and heuristic algorithms. Herein, the innovative tolerant and approximate reducts quantitatively approach/expand/weaken the ideal qualitative reduct. (4) Finally, a decision table example is provided for illustration. This paper performs double-quantitative fusion of causality measures to systematically mine IP-Accuracy, and this measure benignly constructs a granular computing platform and hierarchical reduct system. By resorting to a monotonous uncertainty measure, this study provides an integration-evolution strategy of granular construction for attribute reduction.

An increasing number of common users, in the Internet age, tend to express their emotions on the Web about everything they like or dislike. As a consequence, the number of all kinds of reviews, such as weblogs, production reviews, and news reviews, grows rapidly. This makes it difficult for people to understand the opinions of the reviews and obtain useful emotion information from such a huge number of reviews. Many scientists and researchers have attached more attention to emotion analysis of online information in the natural language processing field. Different from previous works, which just focused on the single-label emotion analysis, this paper takes into account rich and delicate emotions and gives special regard to multi-label emotion recognition for weblog sentences based on the Chinese emotion corpus (Ren-CECps). Using the theory of Bayesian networks and probabilistic graphical model, the latent emotion variable and topic variable are employed to find out the complex emotions of weblog sentences. Our experimental results on the multi-label emotion topic model demonstrate the effectiveness of the model in recognizing the polarity of sentence emotions. © 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

In the field of traffic bottleneck analysis, it is expected to discover traffic congestion patterns from the reports of road conditions. However, data patterns mined by existing KDD algorithms may not coincide with the real application requirements. Different from academic researchers, traffic management officers do not pursue the most frequent patterns but always hold multiple views for mining task to facilitate traffic planning. They expect to study the correlation between traffic congestion and various kinds of road properties, especially the road properties easily to be improved. In this multi-view analysis, each view actually denotes a kind of user preference of road properties. Thus it is required to integrate user-defined attribute preferences into pattern mining process. To tackle this problem, we propose a multi-view attribute reduction model to discover the patterns of user interests. In this model, user views are expressed with attribute preferences and formally represented by attribute orders. Based on this, we implement a workflow of multi-view traffic bottleneck analysis, which consists of data preprocessing, preference representation and congestion pattern mining. We validate our approach based on the reports of road conditions from Shanghai. Experimental results show that the resultant multi-view mining outcomes are effective for analyzing congestion causes and traffic management.

Inconsistent data distributions among multiple views is one of the most crucial aspects of person re-identification. To solve the problem, this paper presents a novel strategy called consistent iterative multi-view transfer learning model. The proposed model captures seven groups of multi-view visual words (MvVW) through an unsupervised cluster method (K-means) from human body. For each group of MvVW, a multi-view discriminative common subspace can be obtained by the fusion of transfer learning and discriminative analysis. In these common subspaces, the original samples can be reconstructed based on MvVW under the low-rank and sparse constraints. Then, we solve it via the inexact augmented Lagrange multiplier method. The proposed strategy is performed on three different challenging person re-identification databases (i.e., VIPeR, CUHK01 and PRID450S), which shows that our model outperforms several state-of-the-art models with improving of 6.36%, 7.7% and 4.0% respectively.

Quantitative attribute reduction exhibits applicability but complexity when compared to qualitative reduction. According to the two-category decision theoretic rough set model, this paper mainly investigates quantitative reducts and their hierarchies (with qualitative reducts) from a regional perspective. (1) An improved type of classification regions is proposed, and its preservation reduct (CRP-Reduct) is studied. (2) Reduction targets and preservation properties of set regions are analyzed, and the set-region preservation reduct (SRP-Reduct) is studied. (3) Separability of set regions and rule consistency is verified, and the quantitative and qualitative double-preservation reduct (DP-Reduct) is established. (4) Hierarchies of CRP-Reduct, SRP-Reduct, and DP-Reduct are explored with two qualitative reducts: the Pawlak-Reduct and knowledge-preservation reduct (KP-Reduct). (5) Finally, verification experiments are provided. CRP-Reduct, SRP-Reduct, and DP-Reduct expand layer by layer Pawlak-Reduct and exhibit quantitative applicability, and the experimental results indicate their effectiveness and hierarchies regarding Pawlak-Reduct and KP-Reduct.

Attribute reduction is the key technique for knowledge acquisition in rough set theory. However, it is still a challenging task to perform attribute reduction on massive data. During the process of attribute reduction on massive data, the key to improving the reduction efficiency is the effective computation of equivalence classes and attribute significance. Aiming at this problem, we propose several parallel attribute reduction algorithms in this paper. Specifically, we design a novel structure of 〈 key , value 〉 pair to speed up the computation of equivalence classes and attribute significance and parallelize the traditional attribute reduction process based on MapReduce mechanism. The different parallelization strategies of attribute reduction are also compared and analyzed from the theoretic view. Abundant experimental results demonstrate the proposed parallel attribute reduction algorithms can perform efficiently and scale well on massive data.

Attribute reduction is an essential subject in rough set theory, but because of quantitative extension, it becomes a problem when considering probabilistic rough set (PRS) approaches. The decision-theoretic rough set (DTRS) has a threshold semantics and decision feature and thus becomes a typical and fundamental PRS. Based on reduction target structures, this paper investigates hierarchical attribute reduction for a two-category DTRS and is divided into five parts. (1) The knowledge-preservation property and reduct are explored by knowledge coarsening. (2) The consistency-preservation principle and reduct are constructed by a consistency mechanism. (3) Region preservation is analyzed, and the separability between consistency preservation and region preservation is concluded; thus, the double-preservation principle and reduct are studied. (4) Structure targets are proposed by knowledge structures, and an attribute reduction is further described and simulated; thus, general reducts are defined to preserve the structure targets or optimal measures. (5) The hierarchical relationships of the relevant four targets and reducts are analyzed, and a decision table example is provided for illustration. This study offers promotion, rationality, structure, hierarchy and generalization, and it establishes a fundamental reduction framework for two-category DTRS. The relevant results also provide some new insights into the attribute reduction problem for PRS.

Because precision and grade act as fundamental quantitative information in approximation space, they are used in relative and absolute quantifications, respectively. At present, the double quantification regarding precision and grade is a novel and valuable subject, but quantitative information fusion has become a key problem. Thus, this paper constructs the double-quantitative approximation space of precision and grade (PG-Approx-Space) and tackles the fusion problem using normal logical operations. It further conducts double-quantification studies on granular computing and rough set models. (1) First, for quantitative information organization and storage, we construct space and plane forms of PG-Approx-Space using the Cartesian product, and for quantitative information extraction and fusion, we establish semantics construction and semantics granules of PG-Approx-Space. (2) Second, by granular computing, we investigate three primary granular issues: quantitative semantics, complete system and optimal calculation. Accordingly, six types of fundamental granules are proposed based on the semantic, microscopic and macroscopic descriptions; their semantics, forms, structures, calculations and relationships are studied, and the granular hierarchical structure is achieved. (3) Finally, we investigate rough set models in PG-Approx-Space. Accordingly, model regions are proposed by developing the classical regions, model expansion is systematically analyzed, some models are constructed as their structures are obtained, and a concrete model is provided. Based on the quantitative information architecture, this paper systematically conducts and investigates double quantification and establishes a fundamental and general exploration framework.

Nowadays, multi-label classification methods are of increasing interest in the areas such as text categorization, image annotation and protein function classification. Due to the correlation among the labels, traditional single-label classification methods are not directly applicable to the multi-label classification problem. This paper presents two novel multi-label classification algorithms based on the variable precision neighborhood rough sets, called multi-label classification using rough sets (MLRS) and MLRS using local correlation (MLRS-LC). The proposed algorithms consider two important factors that affect the accuracy of prediction, namely the correlation among the labels and the uncertainty that exists within the mapping between the feature space and the label space. MLRS provides a global view at the label correlation while MLRS-LC deals with the label correlation at the local level. Given a new instance, MLRS determines its location and then computes the probabilities of labels according to its location. The MLRS-LC first finds out its topic and then the probabilities of new instance belonging to each class is calculated in related topic. A series of experiments reported for seven multi-label datasets show that MLRS and MLRS-LC achieve promising performance when compared with some well-known multi-label learning algorithms.

Due to their unsupervised learning nature, analyzing the semantics of clustering schemes can be difficult. Qualitative information such as preference relations may be useful in semantic analysis of clustering process. This paper describes a framework based on preference or dominance relations that helps us qualitatively analyze a clustering scheme. This qualitative interpretation is shown to be useful for combining clustering schemes that are based on different criteria. The qualitative combination can be used to analyze its quantitative counterpart and can also be used instead of the quantitative combination. The paper further extends the framework to accommodate rough set based clustering. The usefulness of the approach is illustrated using a synthetic retail database.

Automatic image annotation is concerned with the task of assigning one or more semantic concepts to a given image. It is a typical multi-label classification problem. This paper presents a novel multi-label classification framework MLNRS based on neighborhood rough sets for automatic image annotation which considers the uncertainty of the mapping from visual feature space to semantic concepts space. Given a new instances, its neighbors in the training set are firstly identified. After that, based on the concept of upper and lower approximations of neighborhood rough sets, all possible labels of the given instance are found. Then, based on the statistical information gained from the label sets of the neighbors, maximum a posteriori (MAP) principle is utilized to determine the label set for the given instance. Experiments completed for three different image datasets show that MLNRS achieves more promising performance in comparison with to some well-known multi-label learning algorithms. We present a multi-label classification framework based on neighborhood rough sets.We consider the uncertainty of the mapping from the visual feature space to the semantic concept space.The presented methods make a contribution to the reducing of the semantic gap.

Forensic electroencephalogram (EEG)-based lie detection has recently begun using the concealed information test (CIT) as a potentially more robust alternative to the classical comparative questions test. The main problem with using CIT is that it requires an objective and fast decision algorithm under the constraint of limited available information. In this study, we developed a simple and feasible hierarchical knowledge base construction and test method for efficient concealed information detection based on objective EEG measures. We describe how a hierarchical feature space was formed and which level of the feature space was sufficient to accurately predict concealed information from the raw EEG signal in a short time. A total of 11 subjects went through an autobiographical paradigm test. A high accuracy of 95.23% in recognizing concealed information with a single EEG electrode within about 20 seconds demonstrates effectiveness of the method.

Convolutional Neural Networks (ConvNets) have a great improvement on the classification performance compared to traditional image classification technologies and become one of the leaders in computer vision. In this paper, we present a Correcting Reliability Level (CRL) supervised three-way decision (3WD) cascade model to implement image classification of mass commodity data. Our model simulates the human decision process by using 3WD to determine “accepted” or “unsure” for the classification result. When judged as “unsure”, CRL will supervise the 3WD and learn more information to make the final prediction. In addition, we introduce a Class Grouping algorithm based on feedback to learn the similarity between classes, which help us to train several expert ConvNets for different types of commodity images. Experimental results show that our model can effectively reduce the classification error rate compared with the base classifier.

Different from English processing, Chinese text processing starts from word segmentation, and the results of word segmentation will influence the outcomes of subsequent processing especially in short text processing. In this paper, we introduce a novel method for Short Text Information Retrieval based Chinese Question Answering. It is developed from the Discernibility Matrix based Rules Acquisition method. Based on the acquired rules, the matching patterns of the training QA pairs can be represented by the reduced attribute words, and the words can also be represented by the QA patterns. Then the attribute words in the test QA pairs can be used to calculate the matching scores. The experimental results show that the proposed representation method of QA patterns has good flexibility to deal with the uncertainty caused by the Chinese word segmentation, and the proposed method has good performance at both MAP and MRR on the test data.

Clustering categorical data arising as an important problem of data mining has recently attracted much attention. In this paper, the problem of unsupervised dimensionality reduction for categorical data is first studied. Based on the theory of rough sets, the attributes of categorical data are decomposed into a number of rough subspaces. A novel clustering ensemble algorithm based on rough subspaces is then proposed to deal with categorical data. The algorithm employs some of rough subspaces with high quality to cluster the data and yields a robust and stable solution by exploiting the resulting partitions. We also introduce a cluster index to evaluate the solution of clustering algorithm for categorical data. Experimental results for selected UCI data sets show that the proposed method produces better results than those obtained by other methods when being evaluated in terms of cluster validity indexes.

Bayesian rough set model (BRSM), as the hybrid development between rough set theory and Bayesian reasoning, can deal with many practical problems which could not be effectively handled by original rough set model. In this paper, the equivalence between two kinds of current attribute reduction models in BRSM for binary decision problems is proved. Furthermore, binary decision problems are extended to multi-decision problems in BRSM. Some monotonic measures of approximation quality for multi-decision problems are presented, with which attribute reduction models for multi-decision problems can be suitably constructed. What is more, the discernibility matrices associated with attribute reduction for binary decision and multi-decision problems are proposed, respectively. Based on them, the approaches to knowledge reduction in BRSM can be obtained which corresponds well to the original rough set methodology.

Rough set theory is an effective supervised learning model for labeled data. However, it is often the case that practical problems involve both labeled and unlabeled data, which is outside the realm of traditional rough set theory. In this paper, the problem of attribute reduction for partially labeled data is first studied. With a new definition of discernibility matrix, a Markov blanket based heuristic algorithm is put forward to compute the optimal reduct of partially labeled data. A novel rough co-training model is then proposed, which could capitalize on the unlabeled data to improve the performance of rough classifier learned only from few labeled data. The model employs two diverse reducts of partially labeled data to train its base classifiers on the labeled data, and then makes the base classifiers learn from each other on the unlabeled data iteratively. The classifiers constructed in different reduct subspaces could benefit from their diversity on the unlabeled data and significantly improve the performance of the rough co-training model. Finally, the rough co-training model is theoretically analyzed, and the upper bound on its performance improvement is given. The experimental results show that the proposed model outperforms other representative models in terms of accuracy and even compares favorably with rough classifier trained on all training data labeled.

Methods of fuzzy rule extraction based on rough set theory are rarely reported in incomplete interval-valued fuzzy information systems. This paper deals with such systems. Instead of obtaining rules by attribute reduction, which may have a negative effect on inducting good rules, the objective of this paper is to extract rules without computing attribute reducts. The data completeness of missing attribute values is first presented. Two different approximation methods are then defined. Two algorithms based on the two approximation methods, called MRBFA and MRBBA are proposed for rule extraction. The two algorithms are evaluated by a housing database from UCI. The experimental results show that MRBFA and MRBBA achieve better classification performances than the method based on attribute reduction.

Most clustering algorithms become ineffective when provided with unsuitable parameters or applied to datasets which are composed of clusters with diverse shapes, sizes, and densities. To alleviate these deficiencies, we propose a novel split-and-merge hierarchical clustering method in which a minimum spanning tree (MST) and an MST-based graph are employed to guide the splitting and merging process. In the splitting process, vertices with high degrees in the MST-based graph are selected as initial prototypes, and K-means is used to split the dataset. In the merging process, subgroup pairs are filtered and only neighboring pairs are considered for merge. The proposed method requires no parameter except the number of clusters. Experimental results demonstrate its effectiveness both on synthetic and real datasets.

Methods of fuzzy rule extraction based on rough set theory are rarely reported in incomplete interval-valued fuzzy information systems. Thus, this paper deals with such systems. Instead of obtaining rules by attribute reduction, which may have a negative effect on inducting good rules, the objective of this paper is to extract rules without computing attribute reducts. The data completeness of missing attribute values is first presented. Positive and converse approximations in interval-valued fuzzy rough sets are then defined, and their important properties are discussed. Two algorithms based on positive and converse approximations, namely, mine rules based on the positive approximation (MRBPA) and mine rules based on the converse approximation (MRBCA), are proposed for rule extraction. The two algorithms are evaluated by several data sets from the UC Irvine Machine Learning Repository. The experimental results show that MRBPA and MRBCA achieve better classification performances than the method based on attribute reduction.

Text classification has been recognized as one of the key techniques in organizing digital data. The intuition that each algorithm has its bias data and build a high performance classifier via some combination of different algorithm is a long motivation. In this paper, we proposed a two-level hierarchical algorithm that systematically combines the strength of support vector machine (SVM) and k nearest neighbor (KNN) techniques based on variable precision rough sets (VPRS) to improve the precision of text classification. First, an extension of regular SVM named variable precision rough SVM (VPRSVM), which partitions the feature space into three kinds of approximation regions, is presented. Second, a modified KNN algorithm named restrictive k nearest neighbor (RKNN) is put forward to reclassify texts in boundary region effectively and efficiently. The proposed algorithm overcomes the drawbacks of sensitive to noises of SVM and low efficiency of KNN. Experimental results compared with traditional algorithms indicate that the proposed method can improve the overall performance significantly.

Feature selection is viewed as an important preprocessing step for pattern recognition, machine learning and data mining. Traditional hill-climbing search approaches to feature selection have difficulties to find optimal reducts. And the current stochastic search strategies, such as GA, ACO and PSO, provide a more robust solution but at the expense of increased computational effort. It is necessary to investigate fast and effective search algorithms. Rough set theory provides a mathematical tool to discover data dependencies and reduce the number of features contained in a dataset by purely structural methods. In this paper, we define a structure called power set tree (PS-tree), which is an order tree representing the power set, and each possible reduct is mapped to a node of the tree. Then, we present a rough set approach to feature selection based on PS-tree. Two kinds of pruning rules for PS-tree are given. And two novel feature selection algorithms based on PS-tree are also given. Experiment results demonstrate that our algorithms are effective and efficient.