Search

The problem of initialization of active learning is considered in this paper. Especially, this paper studies the problem in an imbalanced data scenario, which is called as class-imbalance active learning cold-start. The novel method is two-stage clustering-based active learning cold-start (ALCS). In the first stage, to separate the instances of minority class from that of majority class, a multi-center clustering is constructed based on a new inter-cluster tightness measure, thus the data is grouped into multiple clusters. Then, in the second stage, the initial training instances are selected from each cluster based on an adaptive candidate representative instances determination mechanism and a clusters-cyclic instance query mechanism. The comprehensive experiments demonstrate the effectiveness of the proposed method from the aspects of class coverage, classification performance, and impact on active learning.

Human activity recognition is a key technology in intelligent video surveillance and an important research direction in the field of computer vision. However, the complexity of human interaction features and the differences in motion characteristics at different time periods have always existed. In this paper, a human interaction recognition algorithm based on parallel multi-feature fusion network is proposed. First of all, in view of the different amount of information provided by the different time periods of action, an improved time-phased video down sampling method based on Gaussian model is proposed. Second, the Inception module uses different scale convolution kernels for feature extraction. It can improve network performance and reduce the amount of network parameters at the same time. The ResNet module mitigates degradation problem due to increased depth of neural networks and achieves higher classification accuracy. The amount of information provided in the motion video in different stages of motion time is also different. Therefore, we combine the advantages of the Inception network and ResNet to extract feature information, and then we integrate the extracted features. After the extracted features are merged, the training is continued to realize parallel connection of the multi-feature neural network. In this paper, experiments are carried out on the UT dataset. Compared with the traditional activity recognition algorithm, this method can accomplish the recognition tasks of six kinds of interactive actions in a better way, and its accuracy rate reaches 88.9%.

Associative classification is a pattern recognition approach that integrates classification and association rule discovery to build accurate classification models. These models are formed by a collection of contrast patterns that fulfill some restrictions. In this paper, we introduce an experimental comparison of the impact of using different restrictions in the classification accuracy. To the best of our knowledge, this is the first time that such analysis is performed, deriving some interesting findings about how restrictions impact on the classification results. Contrasting these results with previously published papers, we found that their conclusions could be unintentionally biased by the restrictions they used. We found, for example, that the jumping restriction could severely damage the pattern quality in the presence of dataset noise. We also found that the minimal support restriction has a different effect in the accuracy of two associative classifiers, therefore deciding which one is the best depends on the support value. This paper opens some interesting lines of research, mainly in the creation of new restrictions and new pattern types by joining different restrictions.

Intrusion Detection Systems (IDS) are necessary and important tools for monitoring information systems. However they produce a huge quantity of alerts. Alerts correlation is a process that reduces the number of alerts reported by intrusion detection systems. In this paper, we propose a new algorithm for a logical-based alerts correlation approach that integrates: security operator's knowledge and preferences. The representation and the reasoning on these knowledge and preferences are done using a new logic called Instantiated First Order Qualitative Choice Logic (IFO-QCL). Our modeling views an alert as an interpretation which allows us to have an efficient algorithm that performs the correlation process in a polynomial time. This paper also provides experimental results which are achieved on datasets issued from a real monitoring system.

The aim of this paper is to improve the classification performance based on the multiclass imbalanced datasets. In this paper, we introduce a new resampling approach based on Clustering with sampling for Multiclass Imbalanced classification using Ensemble (C-MIEN). C-MIEN uses the clustering approach to create a new training set for each cluster. The new training sets consist of the new label of instances with similar characteristics. This step is applied to reduce the number of classes then the complexity problem can be easily solved by C-MIEN. After that, we apply two resampling techniques (oversampling and undersampling) to rebalance the class distribution. Finally, the class distribution of each training set is balanced and ensemble approaches are used to combine the models obtained with the proposed method through majority vote. Moreover, we carefully design the experiments and analyze the behavior of C-MIEN with different parameters (imbalance ratio and number of classifiers). The experimental results show that C-MIEN achieved higher performance than state-of-the-art methods.

One of the challenging problems in artificial intelligence is credit assignment which simply means distributing the credit among a group, such as a group of agents. We made an attempt to meet this problem with the aid of the reinforcement learning paradigm. In this paper, expertness framework is defined and applied to the multi-agent credit assignment problem. In the expertness framework, the critic agent, who is responsible for distributing credit among agents, is equipped with learning capability, and the proposed credit assignment solution is based on the critic to learn to assign a proportion of the credit to each agent, and the used proportion should be learned by reinforcement learning. The paper also reports the degree of expertness framework robustness and the amount of performance decline in noisy environments. Experimental results show the superiority of the method over the common methods of credit assignment used in lots of different domains and also show that performance reduction with respect to the quantity of the noise is tolerable and the system ultimately converges to the stable and correct behavior, therefore the agents are still capable of efficiently performing in the noisy environments.

This paper extends the previous work in the connection between fuzzy classifiers and kernel machines [2] to a general case. In [2], all membership functions for the same input variable are generated from location transformation of a reference function. A translation invariant kernel is constructed from reference functions. The kernel is a Mercer kernel if the reference functions are positive definite. A support vector learning approach for positive definite fuzzy classifiers PDFCs was proposed. In this paper, a smooth support vector learning algorithm for fuzzy rule-based classification systems is proposed. The smooth support vector machine SSVM is capable of generating nonlinear separating surfaces using arbitrary kernels. The positive definiteness requirement on reference functions is relaxed. A fuzzy classifier using arbitrary reference functions can be built from the training samples based on an SSVM. The resulting fuzzy classifier is called standard binary fuzzy classifier SBFC. Fuzzy rules are extracted with each rule given by a training sample. The reduced kernel technique is introduced to simplify the decision function of the SBFC and to reduce the number of fuzzy rules. Finally, the performance of SBFCs with different reference functions is illustrated by experimental results. Comparisons with PDFCs are also provided.

Nowadays, object recognition based on local invariant features is widely acknowledged as one of the best paradigms for object recognition due to its robustness for solving image matching across different views of a given scene. This paper proposes a new approach for learning invariant region descriptor operators through genetic programming and introduces another optimization method based on a hill-climbing algorithm with multiple re-starts. The approach relies on the synthesis of mathematical expressions that extract information derived from local image patches called local features. These local features have been previously designed by human experts using traditional representations that have a clear and, preferably mathematically, well-founded definition. We propose in this paper that the mathematical principles that are used in the description of such local features could be well optimized using a genetic programming paradigm. Experimental results confirm the validity of our approach using a widely accepted testbed that is used for testing local descriptor algorithms. In addition, we compare our results not only against three state-of-the-art algorithms designed by human experts, but also, against a simpler search method for automatically generating programs such as hill-climber. Furthermore, we provide results that illustrate the performance of our improved SIFT algorithms using an object recognition application for indoor and outdoor scenarios.

The use of ROC Receiver Operating Characteristics analysis as a tool for evaluating the performance of classification models in machine learning has been increasing in the last decade. Among the most notable advances in this area are the extension of two-class ROC analysis to the multi-class case as well as the employment of ROC analysis in cost-sensitive learning. Methods now exist which take instance-varying costs into account. The purpose of our paper is to present a survey of this field with the aim of gathering important achievements in one place. In the paper, we present application areas of the ROC analysis in machine learning, describe its problems and challenges and provide a summarized list of alternative approaches to ROC analysis. In addition to presented theory, we also provide a couple of examples intended to illustrate the described approaches.

The work described in this paper addresses the study of association rules within groups of individuals. The analysis of the characteristics and the behavior of the individuals belonging to such groups in a given database is powerful in practice, since it provides a mechanism to deal with groups rather than isolated individuals. In this paper, we define group association rules and we study interestingness measures for them. These interestingness measures can be used to rank, not only groups of individuals, but also rules within each group. We also compare the rankings provided by those different interestingness measures in order to determine which one provides a better alternative depending on the kind of situations we wish to highlight within large databases with many different and overlapping groups of individuals.

Blending is a well-established technique, commonly used to increase performance of predictive models. Its effectiveness has been confirmed in practice as most of the latest international data-mining contest winners were using some kind of a committee of classifiers to produce their final entry. This paper presents a method of using a genetic algorithm to optimize an ensemble of multiple classification or regression models. An implementation of that method in R system, called Genetic Meta-Blender, was tested during the Australasian Data Mining 2009 Analytic Challenge. A subject of this data mining competition was the methods for combining predictive models. The described approach was awarded with the Grand Champion prize for achieving the best overall result. In this paper, the purpose of the challenge is described and details of the winning approach are given. The results of Genetic Meta-Blender are also discussed and compared to several baseline scores. Additionally, GMB is evaluated on data from a different data mining competition, namely SIAM SDM'11 Contest: Prediction of Biological Properties of Molecules from Chemical Structure.

Long-term travel time prediction TTP can be an important planning tool for both freight transport and public transport companies. In both cases it is expected that the use of long-term TTP can improve the quality of the planned services by reducing the error between the actual and the planned travel times. However, for reasons that we try to stretch out along this paper, long-term TTP is almost not mentioned in the scientific literature. In this paper we discuss the relevance of this study and compare three non-parametric state-of-the-art regression methods: Projection Pursuit Regression PPR, Support Vector Machine SVM and Random Forests RF. For each one of these methods we study the best combination of input parameters. We also study the impact of different methods for the pre-processing tasks feature selection, example selection and domain values definition in the accuracy of those algorithms. We use bus travel time's data from a bus dispatch system. From an off-the-shelf point-of-view, our experiments show that RF is the most promising approach from the three we have tested. However, it is possible to obtain more accurate results using PPR but with extra pre-processing work, namely on example selection and domain values definition.

In this paper, a method for telecommunications fraud detection is proposed. The method is based on the user profiling using the Latent Dirichlet Allocation LDA. Fraudulent behavior is detected with use of a threshold-type classification algorithm, allocating the telecommunication accounts into one of two classes: fraudulent account and non-fraudulent account. The paper provides also a method for automatic threshold computation. The accounts are classified with use of the Kullback-Leibler divergence KL-divergence. Therefore, we also introduce three methods for approximating the KL-divergence between two LDAs. Finally, the results of experimental study on KL-divergence approximation and fraud detection in telecommunications are reported.

Understanding the molecular mechanism that govern drug protein interactions is essential for efficient drug design. The progress in drug development is very slow compared to the rising variation in human genomes and the discovery of new diseases. Thus the need to have more efficient and effective drug discovery pipelines is becoming essential. In this paper, we study and analyze the relationships between drugs and proteins that they target. We consider some properties of the proteins that can be used to give weight to protein-protein relationships. We aim to identify protein properties that might guide the drug to proteins. Amino acid enrichment in target clusters is analyzed to assess if certain drugs prefer particular amino acids. The correlation between the net charge of the drug with the amino acids in the target protein is studies as well. Moreover, characterizing the functional components in drug target clusters is necessary to find drug preference. Sequence motifs and domains, post-translational modification and biological pathways of target proteins are analyzed to understand drug preference. This characterizes the drug target proteins from sequence and functional angles. Finally, we realized the importance of the social network model in analyzing any problem that can be modeled as a network. Fortunately, the problem tackled in this paper fits well the network requirement of the social network model. Hence, we analyze the correlations using the social network model where actors are drugs and proteins; our aim is to analyze the relations between drugs and proteins by benefiting from the rich metrics developed to analyze social networks. In this paper, we briefly mention the social network technique in order to demonstrate its applicability which will be detailed in a future publication.

Attribute selection is a data preprocessing step which aims at identifying relevant attributes for the target machine learning task --namely classification in this paper. In this paper, we propose a new attribute selection strategy --based on a lazy learning approach --which postpones the identification of relevant attributes until an instance is submitted for classification. Our strategy relies on the hypothesis that taking into account the attribute values of an instance to be classified may contribute to identifying the best attributes for the correct classification of that particular instance. Experimental results using the k-NN and Naive Bayes classifiers, over 40 different data sets from the UCI Machine Learning Repository and five large data sets from the NIPS 2003 feature selection challenge, show the effectiveness of delaying attribute selection to classification time. The proposed lazy technique in most cases improves the accuracy of classification, when compared with the analogous attribute selection approach performed as a data preprocessing step. We also propose a metric to estimate when a specific data set can benefit from the lazy attribute selection approach.

The accuracy of methods for the assessment of mammographic risk analysis is heavily related to breast tissue characteristics. Previous work has demonstrated considerable success in developing an automatic breast tissue classification methodology which overcomes this difficulty. This paper proposes a unified approach for the application of a number of rough and fuzzy-rough set methods to the analysis of mammographic data. Indeed this is the first time that fuzzy-rough approaches have been applied to this particular problem domain. In the unified approach detailed here feature selection methods are employed for dimensionality reduction developed using rough sets and fuzzy-rough sets. A number of classifiers are then used to examine the data reduced by the feature selection approaches and assess the positive impact of these methods on classification accuracy. Additionally, this paper also employs a new fuzzy-rough classifier based on the nearest neighbour classification algorithm. The novel use of such an approach demonstrates its efficiency in improving classification accuracy for mammographic data, as well as considerably removing redundant, irrelevant, and noisy features. This is supported with experimental application to two well-known datasets. The overall result of employing the proposed unified approach is that feature selection can identify only those features which require extraction. This can have the positive effect of increasing the risk assessment accuracy rate whilst additionally reducing the time required for expert scrutiny, which in-turn means the risk analysis process is potentially quicker and involves less screening.

Recommender systems are changing from novelties used by a few E-commerce sites to serious business tools. They recommend products to customers based on their historical preferences. Through several recommendation techniques, Collaborative filtering (CF) is the most successful recommendation method which is widely used. Nowadays, customer lifetime value (CLV) is measured by RFM (Recency, Frequency, and Monetary) and weighted RFM-based method is used in product recommendation. In this paper, we present a product recommendation technique for online retail stores which employs CLV concept and integrates it with CF method to generate better quality recommendations. In this paper, CF is applied to customer ratings on products, which are collected implicitly by web usage mining approach. Product taxonomy is also used to segment products according to their categories and to reduce dimensions of computational space. The experimental results show that the proposed technique outperforms several other similar recommendation methods.

A video stream is usually massive in terms of data content with abundant information. In the past, extracting explicit semantic information from a video stream; i.e. object detection, object tracking and information extraction; has been extensively investigated. However, little work has been devoted on the problem of discovering global or implicit information from huge video streams. In this paper, a framework has been presented for extracting information for a specified player from soccer video broadcast by data mining techniques. Concepts and information which exist in a soccer video broadcast are useful for team coaches. But, due to various reasons; i.e. wide field of view of a video stream, huge data, existence of great number of important objects in the play field of a soccer match and the occurrence of number of important events, manual extraction of information from soccer video broadcast is difficult and time consuming task. In this paper, a set of techniques is presented that automatically extract some useful information of a player, i.e. velocity and traversed distance, from a soccer video broadcast. Processing of video sequence under change of lighting conditions, fast camera movement and player`s occlusion is a challenging task. Our proposed framework comprise of 3 stages, player segmentation, player tracking and information extraction. All three stages must be robust under various challenges. The performance of our proposed system has been evaluated using a variety of soccer video broadcast having different characteristics in term of lighting conditions. The experiments showed that the efficiency of our system is satisfactory.

In Machine Learning, estimation of the predictive accuracy for a given model is most commonly approached by analyzing the average accuracy of the model. In general, the predictive models do not provide accuracy estimates for their individual predictions. The reliability estimates of individual predictions require the analysis of various model and instance properties. In the paper we make an overview of the approaches for estimation of individual prediction reliability. We start by summarizing three research fields, that provided ideas and motivation for our work: (a) approaches to perturbing learning data, (b) the usage of unlabeled data in supervised learning, and (c) the sensitivity analysis. The main part of the paper presents two classes of reliability estimation approaches and summarizes the relevant terminology, which is often used in this and related research fields.

Often the best model to solve a real-world problem is relatively complex. This paper presents oracle learning, a method using a larger model as an oracle to train a smaller model on unlabeled data in order to obtain (1) a smaller acceptable model and (2) improved results over standard training methods on a similarly sized smaller model. In particular, this paper looks at oracle learning as applied to multi-layer perceptrons trained using standard backpropagation. Using multi-layer perceptrons for both the larger and smaller models, oracle learning obtains a 15.16% average decrease in error over direct training while retaining 99.64% of the initial oracle accuracy on automatic spoken digit recognition with networks on average only 7% of the original size. For optical character recognition, oracle learning results in neural networks 6% of the original size that yield a 11.40% average decrease in error over direct training while maintaining 98.95% of the initial oracle accuracy. Analysis of the results suggest oracle learning is especially appropriate when either the size of the final model is relatively small or when the amount of available labeled data is small.

Analyzing the temporal behaviors and revealing the hidden rules of objects that produce time series data to detect the events that users are interested in have recently received a large amount of attention. Generally, in various application scenarios and most research works, the equal interval sampling of a time series is a requirement. However, this requirement is difficult to guarantee because of the presence of sampling errors in most situations. In this paper, a multigranularity event detection method for an unequal interval time series, called SSED (self-adaptive segmenting based event detection), is proposed. First, in view of the trend features of a time series, a self-adaptive segmenting algorithm is proposed to divide a time series into unfixed-length segmentations based on the trends. Then, by clustering the segmentations and mapping the clusters to different identical symbols, a symbol sequence is built. Finally, based on unfixed-length segmentations, the multigranularity events in the discrete symbol sequence are detected using a tree structure. The SSED is compared to two previous methods with ten public datasets. In addition, the SSED is applied to the public transport systems in Xiamen, China, using bus-speed time-series data. The experimental results show that the SSED can achieve higher efficiency and accuracy than existing algorithms.

Most of real-world datasets are of mixed type including both numeric and categorical attributes. Unlike numbers, operations on categorical values are limited, and the degree of similarity between distinct values cannot be measured directly. In order to properly analyze mixed-type data, dedicated methods to handle categorical values in the datasets are needed. The limitation of most existing methods is lack of appropriate numeric representations of categorical values. Consequently, some of analysis algorithms cannot be applied. In this paper, we address this deficiency by transforming categorical values to their numeric representation so as to facilitate various analyses of mixed-type data. In particular, the proposed transformation method preserves semantics of categorical values with respect to the other values in the dataset, resulting in better performance on data analyses including classification and clustering. The proposed method is verified and compared with other methods on extensive real-world datasets.

Classifier combination through ensemble systems is one of the most effective approaches to improve the accuracy of classification systems. Ensemble systems are generally used to combine classifiers; However, selecting the best combination of individual classifiers is a challenging task. In this paper, we propose an efficient assembling method that employs both meta-learning and a genetic algorithm for the selection of the best classifiers. Our method is called MEGA, standing for using MEta-learning and a Genetic Algorithm for algorithm recommendation. MEGA has three main components: Training, Model Interpretation and Testing. The Training component extracts meta-features of each training dataset and uses a genetic algorithm to discover the best classifier combination. The Model Interpretation component interprets the relationships between meta-features and classifiers using a priori and multi-label decision tree algorithms. Finally, the Testing component uses a weighted k-nearest-neighbors algorithm to predict the best combination of classifiers for unseen datasets. We present extensive experimental results that demonstrate the performance of MEGA. MEGA achieves superior results in a comparison of three other methods and, most importantly, is able to find novel interpretable rules that can be used to select the best combination of classifiers for an unseen dataset.

Most density-based clustering algorithms have the problems of difficult parameter setting, high time complexity, poor noise recognition, and weak clustering for datasets with uneven density. To solve these problems, this paper proposes FOP-OPTICS algorithm (Finding of the Ordering Peaks Based on OPTICS), which is a substantial improvement of OPTICS (Ordering Points To Identify the Clustering Structure). The proposed algorithm finds the demarcation point (DP) from the Augmented Cluster-Ordering generated by OPTICS and uses the reachability-distance of DP as the radius of neighborhood eps of its corresponding cluster. It overcomes the weakness of most algorithms in clustering datasets with uneven densities. By computing the distance of the k-nearest neighbor of each point, it reduces the time complexity of OPTICS; by calculating density-mutation points within the clusters, it can efficiently recognize noise. The experimental results show that FOP-OPTICS has the lowest time complexity, and outperforms other algorithms in parameter setting and noise recognition.

In technical systems the analysis of similar load situations is a promising technique to gain information about the system’s state, its health or wearing. Very often, load situations are challenging to be defined by hand. Hence, these situations need to be discovered as recurrent patterns within multivariate time series data of the system under consideration. Unsupervised algorithms for finding such recurrent patterns in multivariate time series must be able to cope with very large data sets because the system might be observed over a very long time. In our previous work we identified discretization-based approaches to be very interesting for variable length pattern discovery because of their low computing time due to the simplification (symbolization) of the time series. In this paper we propose additional preprocessing steps for symbolic representation of time series aiming for enhanced multivariate pattern discovery. Beyond that we show the performance (quality and computing time) of our algorithms in a synthetic test data set as well as in a real life example with 100 millions of time points. We also test our approach with increasing dimensionality of the time series.

In the last ten years, many variants of the principal component analysis were suggested to fight against the curse of dimensionality. Recently, A. Sharma et al. have proposed a stable numerical algorithm based on Householder QR decomposition (HQR) called QR PCA. This approach improves the performance of the PCA algorithm via a singular value decomposition (SVD) in terms of computation complexity. In this paper, we propose a new algorithm called RRQR PCA in order to enhance the QR PCA performance by exploiting the Rank-Revealing QR Factorization (RRQR). We have also improved the recognition rate of RRQR PCA by developing a nonlinear extension of RRQR PCA. In addition, a new robust RBF Lp-norm kernel is proposed in order to reduce the effect of outliers and noises. Extensive experiments on two well-known standard face databases which are ORL and FERET prove that the proposed algorithm is more robust than conventional PCA, 2DPCA, PCA-L1, WTPCA-L1, LDA, and 2DLDA in terms of face recognition accuracy.

Overlapping communities exist in real networks, where the communities represent hierarchical community structures, such as schools and government departments. A non-binary tree allows a vertex to belong to multiple communities to obtain a more realistic overlapping community structure. It is challenging to select appropriate leaf vertices and construct a hierarchical tree that considers a large amount of structural information. In this paper, we propose a non-binary hierarchical tree overlapping community detection based on multi-dimensional similarity. The multi-dimensional similarity fully considers the local structure characteristics between vertices to calculate the similarity between vertices. First, we construct a similarity matrix based on the first and second-order neighbor vertices and select a leaf vertex. Second, we expand the leaf vertex based on the principle of maximum community density and construct a non-binary tree. Finally, we choose the layer with the largest overlapping modularity as the result of community division. Experiments on real-world networks demonstrate that our proposed algorithm is superior to other representative algorithms in terms of the quality of overlapping community detection.

Biological network classification is an eminently challenging task in the domain of data mining since the networks contain complex structural information. Conventional biochemical experimental methods and the existing intelligent algorithms still suffer from some limitations such as immense experimental cost and inferior accuracy rate. To solve these problems, in this paper, we propose a novel framework for Biological graph classification named Biogc, which is specifically developed to predict the label of both small-scale and large-scale biological network data flexibly and efficiently. Our framework firstly presents a simplified graph kernel method to capture the structural information of each graph. Then, the obtained informative features are adopted to train different scale biological network data-oriented classifiers to construct the prediction model. Extensive experiments on five benchmark biological network datasets on graph classification task show that the proposed model Biogc outperforms the state-of-the-art methods with an accuracy rate of 98.90% on a larger dataset and 99.32% on a smaller dataset.

Wavelet analysis is a practical tool to study signal analysis and image processing. Traditional Fourier transform can also transfer the signal into frequency domain, but wavelet analysis is more attractive for its features of multi-resolution and localization of frequency. Recently, there has been significant development in the use of wavelet methods in the data mining process. However, the objective of the study described in this paper is twofold: designing a wavelet transform algorithm on the multiprocessor architecture and using this algorithm in mining spatial outliers of meteorological data. Spatial outliers are the spatial objects with distinct features from their surrounding neighbors. Outlier detection reveals important and valuable information from large spatial data sets. As region outliers are commonly multi-scale objects, wavelet analysis is an effective tool to study them. In this paper, we present a wavelet based approach and its applicability in outlier detection. We design a suite of algorithms to effectively discover region outliers and also a parallel algorithm is designed to bring efficiency and speedup for the wavelet analysis. The applicability and effectiveness of the developed algorithms are evaluated on real-world meteorological dataset.

An important area of data mining is anomaly detection, particularly for fraud. However, little work has been done in terms of detecting anomalies in data that is represented as a graph. In this paper we present graph-based approaches to uncovering anomalies in domains where the anomalies consist of unexpected entity/relationship alterations that closely resemble non-anomalous behavior. We have developed three algorithms for the purpose of detecting anomalies in all three types of possible graph changes: label modifications, vertex/edge insertions and vertex/edge deletions. Each of our algorithms focuses on one of these anomalous types, using the minimum description length principle to first discover the normative pattern. Once the common pattern is known, each algorithm then uses a different approach to discover particular anomalous types. In this paper, we validate all three approaches using synthetic data, verifying that each of the algorithms on graphs and anomalies of varying sizes, are able to detect the anomalies with very high detection rates and minimal false positives. We then further validate the algorithms using real-world cargo data and actual fraud scenarios injected into the data set with 100% accuracy and no false positives. Each of these algorithms demonstrates the usefulness of examining a graph-based representation of data for the purposes of detecting fraud.

The presence of a substantial number of noisy instances in a given dataset may adversely affect the hypothesis learnt from that data. Removing noisy instances prior to the construction of a classifier has been shown to improve the classification ability of a learner on new data. This paper introduces a novel technique for identifying observations with class noise in a dataset using frequent itemsets. For the given dataset, each instance is assigned a NoiseFactor, indicating a relative likelihood that it contains class noise. A frequent itemset is a set of instances with common attribute values which contains at least as many instances as a user-defined minimum support threshold. Consequently, the set of frequent itemsets contains information related to the structure and dependence between the attributes. Each frequent itemset is assigned a class, based on the proportion of instances within the itemset from each class. Instances that are contained in itemsets that have a large proportion of instances from the other class are identified as noisy. The technique proposed in this paper is analyzed in numerous case studies using real-world software measurement datasets with either inherent or injected noise. A comparison is provided with two well-known techniques for the identification of class noise: Classification Filter and Ensemble Filter. The results demonstrate that this new algorithm is very effective at identifying instances with class noise.

The analysis of microarray data from time-series experiments requires specialised algorithms, which take the temporal ordering of the data into account. In this paper we explore a new architecture of Bayesian classifier that can be used to understand how biological mechanisms differ with respect to time. We show that this classifier improves the classification of microarray data and at the same time ensures that the models can easily be analysed by biologists by incorporating time transparently. In this paper we focus on data that has been generated to explore different types of muscular dystrophy.

Spectral clustering has been an effective clustering method, in last decades, because it can get an optimal solution without any assumptions on data’s structure. The basic key in spectral clustering is its similarity matrix. Despite many empirical successes in similarity matrix construction, almost all previous methods suffer from handling just one objective. To address the multi-objective ensemble clustering, we introduce a new ensemble manifold regularization (MR) method based on stacking framework. In our Manifold Regularization Ensemble Clustering (MREC) method, several objective functions are considered simultaneously, as a robust method for constructing the similarity matrix. Using it, the unsupervised extreme learning machine (UELM) is employed to find the generalized eigenvectors to embed the data in low-dimensional space. These eigenvectors are then used as the base point in spectral clustering to find the best partitioning of the data. The aims of this paper are to find robust partitioning that satisfy multiple objectives, handling noisy data, keeping diversity-based goals, and dimension reduction. Experiments on some real-world datasets besides to three benchmark protein datasets demonstrate the superiority of MREC over some state-of-the-art single and ensemble methods.

In classification, a decision tree is a common model due to its simple structure and easy understanding. Most of decision tree algorithms assume all instances in a dataset have the same degree of confidence, so they use the same generation and pruning strategies for all training instances. In fact, the instances with greater degree of confidence are more useful than the ones with lower degree of confidence in the same dataset. Therefore, the instances should be treated discriminately according to their corresponding confidence degrees when training classifiers. In this paper, we investigate the impact and significance of degree of confidence of instances on the classification performance of decision tree algorithms, taking the classification and regression tree (CART) algorithm as an example. First, the degree of confidence of instances is quantified from a statistical perspective. Then, a developed CART algorithm named C_CART is proposed by introducing the confidence of instances into the generation and pruning processes of CART algorithm. Finally, we conduct experiments to evaluate the performance of C_CART algorithm. The experimental results show that our C_CART algorithm can significantly improve the generalization performance as well as avoiding the over-fitting problem to a certain extend.

Multiclass data classification, where the goal is to segment data into classes, is an important task in machine learning. However, the task is challenging due to reasons including the scarcity of labeled training data; in fact, most machine learning algorithms require a large amount of labeled examples to perform well. Moreover, the accuracy of a classifier can be dependent on the accuracy of the training labels which can be corrupted. In this paper, we present an efficient and unconditionally stable semi-supervised graph-based method for multiclass data classification which requires considerably less labeled training data to accurately classify a data set compared to current techniques, due to properties such as the embedding of data into a similarity graph. In particular, it performs very well and more accurately than current approaches in the common scenario of few labeled training elements. Morever, we show that the algorithm performs with good accuracy even with a large number of mislabeled examples and is also able to incorporate class size information. The proposed method uses a modified auction dynamics technique. Extensive experiments on benchmark datasets are performed and the results are compared to other methods.

The construction of fuzzy rule-based classification systems with both good generalization ability and interpretability is a chalenging issue. The paper aims to present a novel framework for the realization of these important (and many times conflicting) goals simultaneously. The generalization performance is obtained with the adaptation of Support Vector algorithms for the identification of a Support Vector Fuzzy Inference (SVFI) system. The SVFI is a fuzzy inference system that implements the Support Vector network inference and inherits from it the robust learning potential. The construction of the SVFI is based on the algorithms presented in [15]. The contribution of the paper is the development of algorithms for the construction of interpretable rule systems on top of the SVFI system. However, the SVFI rules usually lack interpretability. For this reason, the accurate set of rules can be approximated with a simpler interpretable fuzzy system that can present insight to the more important aspects of the data. Therefore, the presented approach utilizes two sets of fuzzy rules: the accurate Support Vector Fuzzy Inference (SVFI) rules and the approximate interpretable one that is derived from the SVFI with a set of tunable threshold parameters. The paper initially reviews the peculiarities of extracting fuzzy rules with Support Vector learning for classification problems. The interpretable fuzzy system construction algorithms receive an a priori description of a set of fuzzy sets that describe the linguistic aspects of the input variables as they are usually perceived by the human experts. After this specification the presented algorithms extract from the SVFI rules a small and concise set of interpretable rules. We apply our method to both synthetic data, data sets from the UCI repository and real gene expression data and we demonstrate its efficiency in uncovering clear and concise rules in application domains where interpretable linguistic labels can be pre-assigned (as in gene expression analysis).

This paper presents a novel mechanism to analyze and summarize the statistical correlations among the attributes of a data cube. To perform the analysis and summarization, this paper proposes a new measure of statistical significance. The main reason for proposing the new measure of statistical significance is to have an essential closure property, which is exploited in the summarization stage of the data mining process. In addition to the closure property, the proposed measure of statistical significance has two other important properties. First, the proposed measure of statistical significance is more conservative than the well-known chi-square test in classical statistics and, therefore, inherits its statistical robustness. This paper does not simply employ the chi-square test due to lack of the desired closure property, which may lead to a precision problem in the summarization process. The second additional property is that, though the proposed measure of statistical significance is more conservative than the chi-square test, for most cases, the proposed measure yields a value that is almost equal to the z test, a conventional measurement of statistical significance based on the normal distribution. Based on the closure property addressed above, this paper develops an algorithm to summarize the results from performing statistical analysis in the data cube. Though the proposed measure of statistical significance avoids the precision problem due to having the closure property, its conservative nature may lead to a recall rate problem in the data mining process. On the other hand, if the chi-square test, which does not have the closure property, was employed, then the summarization process may suffer a precision problem. In this paper, we also show a possible application in bioinformatics. We applied the proposed mechanism on a microarray dataset, in order to identify groups of genes with similar expression patterns in subspaces of feature space.

Typing rhythms are the rawest form of data stemming from the interaction between users and computers. When properly sampled and analyzed, they may become a useful tool to ascertain personal identity. Moreover, unlike other biometric features, typing dynamics have an important characteristic: they still exist and are available even after an access control phase has been passed. As a consequence, keystroke analysis can be used as a viable tool for user authentication throughout the work session. In this paper we present an original approach to identity verification based on the analysis of the typing rhythms of individuals on different texts. Our experiments involve 130 volunteers and reach the best outcomes found in the literature, using a smaller amount of information than in other works, and avoiding any form of tailoring of the system to the available data set. The method described in the paper is easily tuned to reach an acceptable trade-off between the need to spot most impostors and to avoid false alarms, and, as a consequence, it can become a valid aid to intrusion detection.

This paper addresses the issue of reducing the storage requirements on instance-based learning algorithms. Algorithms proposed by other researches use heuristics to prune instances of the training set or modify the instances themselves to achieve a reduced set of instances. This paper presents an alternative way. The presented approach proposes to induce a reduced set of prototypes (partially-defined instances) with evolutionary algorithms. Experiments were performed with GALE, a fine-grained parallel evolutionary algorithm, and other well-known reduction techniques on several data sets. Results suggest that GALE is competitive and robust for inducing sets of partially-defined instances. Moreover, it achieves better reduction rates in storage requirements without losses in generalization accuracy. Simultaneously, if the partially-defined instances induced by GALE are post-processed, results can also be used for attribute selection.

The difficulty of cross-domain text sentiment classification is that the data distributions in the source domain and the target domain are inconsistent. This paper proposes an attention network based on feature sequences (ANFS) for cross-domain sentiment classification, which focuses on important semantic features by using the attention mechanism. Particularly, ANFS uses a three-layer convolutional neural network (CNN) to perform deep feature extraction on the text, and then uses a bidirectional long short-term memory (BiLSTM) to capture the long-term dependency relationship among the text feature sequences. We first transfer the ANFS model trained on the source domain to the target domain and share the parameters of the convolutional layer; then we use a small amount of labeled target domain data to fine-tune the model of the BiLSTM layer and the attention layer. The experimental results on cross-domain sentiment analysis tasks demonstrate that ANFS can significantly outperform the state-of-the-art methods for cross-domain sentiment classification problems.

In this paper, we present a novel approach for n-gram generation in text classification. The a-priori algorithm is adapted to prune word sequences by combining three feature selection techniques. Unlike the traditional two-step approach for text classification in which feature selection is performed after the n-gram construction process, our proposal performs an embedded feature elimination during the application of the a-priori algorithm. The proposed strategy reduces the number of branches to be explored, speeding up the process and making the construction of all the word sequences tractable. Our proposal has the additional advantage of constructing a low-dimensional dataset with only the features that are relevant for classification, that can be used directly without the need for a feature selection step. Experiments on text classification datasets for sentiment analysis demonstrate that our approach yields the best predictive performance when compared with other feature selection approaches, while also facilitating a better understanding of the words and phrases that explain a given task; in our case online reviews and ratings in various domains.

Early and precise diagnosis of schizophrenia disorder (SZ) has an essential role in the quality of a patient’s life and future treatments. Structural and functional neuroimaging provides robust biomarkers for understanding the anatomical and functional changes associated with SZ. Each of the neuroimaging techniques shows only a different perspective on the functional or structural of the brain, while multi-modal fusion can reveal latent connections in the brain. In this paper, we propose an approach for the fusion of structural and functional brain data with a deep learning-based model to take advantage of data fusion and increase the accuracy of schizophrenia disorder diagnosis. The proposed method consists of an architecture of 3D convolutional neural networks (CNNs) that applied to magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and diffusion tensor imaging (DTI) extracted features. We use 3D MRI patches, fMRI spatial independent component analysis (ICA) map, and DTI fractional anisotropy (FA) as model inputs. Our method is validated on the COBRE dataset, and an average accuracy of 99.35% is obtained. The proposed method demonstrates promising classification performance and can be applied to real data.

Bagging has attracted much attention due to its simple implementation and the popularity of bootstrapping. By learning diverse classifiers from resampled datasets and averaging the outcomes, bagging investigates the possibility of achieving substantial classification performance of the base classifier. Diversity has been recognized as a very important characteristic in bagging. This paper presents an efficient and effective bagging approach, that learns a set of independent Bayesian network classifiers (BNCs) from disjoint data subspaces. The number of bits needed to describe the data is measured in terms of log likelihood, and redundant edges are identified to optimize the topologies of the learned BNCs. Our extensive experimental evaluation on 54 publicly available datasets from the UCI machine learning repository reveals that the proposed algorithm achieves a competitive classification performance compared with state-of-the-art BNCs that use or do not use bagging procedures, such as tree-augmented naive Bayes (TAN), k-dependence Bayesian classifier (KDB), bagging NB or bagging TAN.

Storyline extraction aims to generate concise summaries of related events unfolding over time from a collection of temporally-ordered news articles. Some existing approaches to storyline extraction are typically built on probabilistic graphical models that jointly model the extraction of events and the storylines from news published in different periods. However, their parameter inference procedures are often complex and require a long time to converge, which hinders their use in practical applications. More recently, a neural network-based approach has been proposed to tackle such limitations. However, event representations of documents, which are important for the quality of the generated storylines, are not learned. In this paper, we propose a novel unsupervised neural network-based approach to extract latent events and link patterns of storylines jointly from documents over time. Specifically, event representations are learned by a stacked autoencoder and clustered for event extraction, then a fusion component is incorporated to link the related events across consecutive periods for storyline extraction. The proposed model has been evaluated on three news corpora and the experimental results show that it outperforms state-of-the-art approaches with significant improvements.

Oversampling ratio N and the minority class’ nearest neighboring number k are key hyperparameters of synthetic minority oversampling technique (SMOTE) to reconstruct the class distribution of dataset. No optimal default value exists there. Therefore, it is of necessity to discuss the influence of the output dataset on the classification performance when SMOTE adopts various hyperparameter combinations. In this paper, we propose a hyperparameter optimization algorithm for imbalanced data. By iterating to find reasonable N and k for SMOTE, so as to build a balanced and high-quality dataset. As a result, a model with outstanding performance and strong generalization ability is trained, thus effectively solving imbalanced classification. The proposed algorithm is based on the hybridization of simulated annealing mechanism (SA) and particle swarm optimization algorithm (PSO). In the optimization, Cohen’s Kappa is used to construct the fitness function, and AdaRBFNN, a new classifier, is integrated by multiple trained RBF neural networks based on AdaBoost algorithm. Kappa of each generation is calculated according to the classification results, so as to evaluate the quality of candidate solution. Experiments are conducted on seven groups of KEEL datasets. Results show that the proposed algorithm delivers excellent performance and can significantly improve the classification accuracy of the minority class.

Network Embedding (NE) has emerged as a powerful tool in many applications. Many real-world networks have multiple types of relations between the same entities, which are appropriate to be modeled as multiplex networks. However, at random walk-based embedding study for multiplex networks, very little attention has been paid to the problems of sampling bias and imbalanced relation types. In this paper, we propose an Adaptive Node Embedding Framework (ANEF) based on cross-layer sampling strategies of nodes for multiplex networks. ANEF is the first framework to focus on the bias issue of sampling strategies. Through metropolis hastings random walk (MHRW) and forest fire sampling (FFS), ANEF is less likely to be trapped in local structure with high degree nodes. We utilize a fixed-length queue to record previously visited layers, which can balance the edge distribution over different layers in sampled node sequence processes. In addition, to adaptively sample the cross-layer context of nodes, we also propose a node metric called Neighbors Partition Coefficient (NPC). Experiments on real-world networks in diverse fields show that our framework outperforms the state-of-the-art methods in application tasks such as cross-domain link prediction and mutual community detection.

Quantifying the abnormal degree of each instance within data sets to detect outlying instances, is an issue in unsupervised anomaly detection research. In this paper, we propose a robust anomaly detection method based on principal component analysis (PCA). Traditional PCA-based detection algorithms commonly obtain a high false alarm for the outliers. The main reason is that ignores the difference of location and scale to each component of the outlier score, this leads to the cumulated outlier score deviates from the true values. To address the issue, we introduce the median and the Median Absolute Deviation (MAD) to rescale each outlier score that mapped onto the corresponding principal direction. And then, the true outlier scores of instances can be obtained as the sum of weighted squares of the rescaled scores. Also, the issue that the assignment of the weight for each outlier score will be solved. The main advantage of our new approach is easy to build with unsupervised data and the recognition performance is better than the classical PCA-based methods. We compare our method to the five different anomaly detection techniques, including two traditional PCA-based methods, in our experiment analysis. The experimental results show that the proposed method has a good performance for effectiveness, efficiency, and robustness.

A safe operation protocol of the wind blades is a critical factor to ensure the stability of a wind turbine. Sensors are most commonly applied for defect detection on wind turbine blades (WTBs). However, due to the high cost and the sensitivity to stochastic noise, computer vision-guided automatic detection remains a challenge for surface defect detection on WTBs in particularly, its accuracy in locating defects is yet to be optimized. In this paper, we developed a visual inspection model that can automatically and precisely classify and locate the surface defects, through the utilization of a deep learning framework based on the Cascade R-CNN. In order to obtain high mean average precision (mAP) according to the characteristics of the dataset, a model named Contextual Aligned-Deformable Cascade R-CNN (CAD Cascade R-CNN) using improved strategies of transfer learning, Deformable Convolution and Deformable RoI Align, as well as context information fusion is proposed and a dataset with surface defects categorized and labeled as crack, breakage and oil pollution is generated. Moreover to alleviate the problem of false detection under a complex background, an improved bisecting k-means is presented during the test process. The adaptability and generalization of the proposed CAD Cascade R-CNN model were validated by each type of defects in dataset and different IoU thresholds, whereas, each of the above improved strategies was verified by gradual ablation experiments. Finally experiments that compared with the baseline Cascade R-CNN, Faster R-CNN and YOLO-v3 demonstrate its superiority over these existing approaches with a maximum of 92.1% mAP.

Sales forecasting is an important part of e-commerce and is critical to smart business decisions. The traditional forecasting methods mainly focus on building a forecasting model, training the model through historical data, and then using it to forecast future sales. Such methods are feasible and effective for the products with rich historical data while they are not performing as well for the newly listed products with little or no historical data. In this paper, with the idea of collaborative filtering, a similarity-based sales forecasting (S-SF) method is proposed. The implementation framework of S-SF includes three modules in order. The similarity module is responsible for generating top-k similar products of a given new product. We calculate the similarity based on two data types: time series data of sales and text data such as product attributes. In the learning module, we propose an attention-based ConvLSTM model which we called AttConvLSTM, and optimize its loss function with the convex function information entropy. Then AttConvLSTM is integrated with Facebook Prophet model to forecast top-k similar products sales based on their historical data. The prediction results of all top-k similar products will be fused in the forecasting module through operations of alignment and scaling to forecast the target products sales. The experimental results show that the proposed S-SF method can simultaneously adapt to the sales forecasting of mature products and new products, which shows excellent diversity, and the forecasting idea based on similar products improves the accuracy of sales forecasting.