Search

Learning from complementary labels (CLs) is a useful learning paradigm, where the CL specifies the classes that the instance does not belong to, instead of providing the ground truth as in the ordinary supervised learning scenario. In general, although it is less laborious and more efficient to collect CLs compared with ordinary labels, the less informative signal in the complementary supervision is less helpful to learn competent feature representation. Consequently, the final classifier's performance greatly deteriorates. In this article, we leverage generative adversarial networks (GANs) to derive an algorithm GAN-CL to effectively learn from CLs. In addition to the role in original GAN, the discriminator also serves as a normal classifier in GAN-CL, with the objective constructed partly with the complementary information. To further prove the effectiveness of our schema, we study the global optimality of both generator and discriminator for the GAN-CL under mild assumptions. We conduct extensive experiments on benchmark image datasets using deep models, to demonstrate the compelling improvements, compared with state-of-the-art CL learning approaches.

In the context of big data, organizations and individuals can often benefit from the data mining techniques, such as classification. However, decision-makers must quickly react to insights over time under dynamic environments. In this paper, we present a novel perspective, named concept-cognitive computing system (C3S), to achieve dynamic classification learning over the partially labeled data and labeled data. More specifically, to store and consolidate knowledge, a concept falling space is first employed as a basic knowledge memory mechanism in C3S. Then, we design a new concept-cognitive process by means of simulating human learning processes, which can incorporate new information into the old knowledge. Finally, a strategy of constructing two different concept spaces is considered in our system when faced with the scenario of a partially labeled dynamic data. Although there exist significant differences between C3S and the conventional incremental learning methods in the learning paradigm, our proposed C3S still performs strong performance for dynamic classification in comparison with several state-of-the-art incremental learning approaches. In addition, the experiments on various datasets have demonstrated that our system can obtain a good performance on the partially labeled data and labeled data simultaneously in dynamic environments.

In human concept learning, people can naturally combine a handful of labeled data with abundant unlabeled data when they make classification decisions, which is also known as semi-supervised learning (SSL) in machine learning. Especially, human concept learning not only is a static process in human cognition but also can vary gradually with dynamic environments. Nevertheless, the classical SSL algorithms must be redesigned to accommodate newly input data. In this sense, concept-cognitive learning may be a good choice, as it can implement dynamic processes by imitating human cognitive processes. Meanwhile, numerous SSL methods were designed based on the feature vector information of instances, while ignoring concept structural information. Based on this idea, a novel SSL method, named semi-supervised concept learning method (S2CL), is proposed for dynamic SSL by employing concept lattices. Moreover, to make full use of the global and local conceptual information, we further propose an extended version of S2CL for concept learning. More specifically, we first show some new related theories for S2CL based on a regular formal decision context; then a novel SSL framework and its corresponding algorithms are designed. Finally, we conduct some experiments on various datasets to demonstrate the effectiveness of our methods.

Concept-cognitive learning (CCL) is an emerging field of concerning incremental concept learning and dynamic knowledge processing in the context of dynamic environments. Although CCL has been widely researched in theory, the existing studies of CCL have one problem: the concepts obtained by CCL systems do not have generalization ability. In the meantime, the existing incremental algorithms still face some challenges that: 1) classifiers have to adapt gradually and 2) the previously acquired knowledge should be efficiently utilized. To address these problems, based on the advantage that CCL can naturally integrate new data into itself for enhancing flexibility of concept learning, we first propose a new CCL model (CCLM) to extend the classical methods of CCL, which is not only a new classifier but also good at incremental learning. Unlike the existing CCL systems, the theory of CCLM is mainly based on a formal decision context rather than a formal context. In learning concepts from dynamic environments, we show that CCLM can naturally incorporate new data into itself with a sufficient theoretical guarantee for incremental learning. For classification task and knowledge storage, our results on various data sets demonstrate that CCLM can simultaneously: 1) achieve the state-of-the-art static and dynamic classification task and 2) directly accomplish preservation of previously acquired knowledge (or concepts) under dynamic environments.

Single Image Super-Resolution (SISR) is a fundamental and important low-level computer vision (CV) task, yet its performance on real-world applications is not always satisfactory. Different from the previous SISR research, we focus on a specific but realistic SR issue: How can we obtain satisfied SR results from compressed JPG (C-JPG) images, which is a ubiquitous image format to greatly release storage space while missing fine details. the JPG SR task is deeply analyzed to discover the connotation. Then, we propose an effective two-step model structure named RGSR, involving two specifically designed components, i.e., JPG recovering and SR generation, instead of the perspective of noise elimination in traditional SR approaches. Besides, we further integrate the cycle loss to build a hybrid objective across scales for better SR generation. Experimental results on both of the standard test data sets and real images show that our approach achieves outstanding results and succeed in applying to practical C-JPG SR tasks.

In many disciplines, the evaluation of algorithms for processing massive data is a challenging research issue. However, different algorithms can produce different or even conflicting evaluation performance, and this phenomenon has not been fully investigated. The motivation of this paper aims to propose a solution scheme for the evaluation of clustering algorithms to reconcile different or even conflicting evaluation performance. The goal of this research is to propose and develop a model, called decision-making support for evaluation of clustering algorithms (DMSECA), to evaluate clustering algorithms by merging expert wisdom in order to reconcile differences in their evaluation performance for information fusion during a complex decision-making process. The proposed model is tested and verified by an experimental study using six clustering algorithms, nine external measures, and four MCDM methods on 20 UCI data sets, including a total of 18,310 instances and 313 attributes. The proposed model can generate a list of algorithm priorities to produce an optimal ranking scheme, which can satisfy the decision preferences of all the participants. The results indicate our developed model is an effective tool for selecting the most appropriate clustering algorithms for given data sets. Furthermore, our proposed model can reconcile different or even conflicting evaluation performance to reach a group agreement in a complex decision-making environment.

Nowadays, datasets are always dynamic and patterns in them are changing. Instances with different labels are intertwined and often linearly inseparable, which bring new challenges to traditional learning algorithms. This paper proposes adaptive hyper-sphere (AdaHS), an adaptive incremental classifier, and its kernelized version: Nys-AdaHS. The classifier incorporates competitive training with a border zone. With adaptive hidden layer and tunable radii of hyper-spheres, AdaHS has strong capability of local learning like instance-based algorithms, but free from slow searching speed and excessive memory consumption. The experiments showed that AdaHS is robust, adaptive, and highly accurate. It is especially suitable for dynamic data in which patterns are changing, decision borders are complicated, and instances with the same label can be spherically clustered.

To make better use of the cloud computing technology, and to overcome the computing and storage requirements which increase rapidly with the number of training samples, in this paper, a new parallel algorithm is proposed—Parallel Regularized Multiple-Criteria Linear Programming (PRMCLP) algorithm—The RMCLP model is converted into a unconstrained optimization problem, and then, in the parallel version, it is split into several tasks, where each part is mapped and computed on a separate processor. This approach enables us to obtain efficiently the final optimization solution of the whole classification problem. At last, we apply this algorithm to Medical data classification. All experiments show that our method and approach greatly increases the training speed of RMCLP in the parallel case.

Network intrusion detection (NID) is an important cyber security scheme to identify attacks in network traffic. Recent years, a large amount of studies try to improve the accuracy of the NID by kinds of deep learning approaches. However, these models always require a lot of calculation and space, which constitutes a major hurdle to practical implementation of DL-based models. Thus, lightweight model is imperative, but there are very few applications of DL-based lightweight algorithms in NID models. In this paper, we propose a lightweight knowledge distillation (LKD) model for NID using the idea of knowledge distillation and separable convolution. To the best of our knowledge, it is the first system to use the knowledge distillation approach for NID. The experiment results show that the accuracy of the proposed approach reaches 91.46% and 94.30% on the KDD-CUP99 and UNSW-NB15 datasets respectively. The performance of our model is superior to some approaches based on deep neural network or some machine learning methods. Moreover, both the computational cost and model size of our model are reduced by about 99% compared to the original model.

Concept-cognitive learning (CCL) concerns the manner in which to incorporate new information into itself by mimicking human cognitive processes. Numerous CCL systems have been proposed to meet different requirements based on formal concept analysis. However, most of these can only operate effectively on small-scale datasets and lack classification ability. To overcome these two challenges, based on a regular formal decision context and multi-thread technique, a new concurrent CCL model is proposed in this study as an extension of classical CCL. More precisely, to enhance computational efficiency, a new concurrent learning framework is designed and its corresponding learning algorithms at the initial concept construction and CCL stages are developed. We propose a concurrent incremental learning technique by continuously accommodating newly added data to meet the requirements of classification tasks. Finally, experiments on various datasets, including real-world and synthetic data, demonstrate that the proposed concurrent method can achieve comparable classification effectiveness and significantly improve the concept learning performance simultaneously.

Traffic prediction is a complex, nonlinear spatiotemporal relationship modeling task with the randomness of traffic demand, the spatial and temporal dependency between traffic flows, and other recurrent and nonrecurrent factors. Based on the ability to learn generic features from history information, deep learning approaches have been recently applied to traffic prediction. Convolutional neural network (CNN) methods that learn traffic as images can improve the predictive accuracy by leveraging the implicit correlations among nearby links. Traffic prediction based on CNN is still in its initial stage without making full use of spatiotemporal traffic information. In this paper, we improve the predictive accuracy by directly capturing the relationship between the input sequence and the predicted value. We propose the new local receptive fields for spatiotemporal traffic information to provide the constraints in the task domain for CNN which is different from traditionally learning traffic as images. We explore a max-pooled CNN followed by a fully connected layer with a nonlinear activation function to convolute the new local receptive fields. The higher global-level features are fed into a predictor to generate the predicted output. Based on the dataset provided by Highways England, we validate the assumption that there exists direct relationship between the input sequence and the predicted value. We train the proposed method by using the backpropagation approach, and we employ the AdaGrad method to update the parameters of the proposed method. The experimental results show that the proposed method can improve the predictive accuracy.

Text is a typical example of unstructured and heterogeneous data in which massive useful knowledge is embedded. Sentiment analysis is used to analyze and predict sentiment polarities of the text. This paper provides a survey and gives comparative analyses of the latest articles and techniques pertaining to lexicon-based, traditional machine learning-based, deep learning-based, and hybrid sentiment analysis approaches. These approaches have their own superiority and get the state-of-the-art results on diverse sentiment analysis tasks. Besides, a brief sentiment analysis example in the tourism domain is displayed, illustrating the entire process of sentiment analysis. Furthermore, we create a large table to compare the pros and cons of different types of approaches, and discuss some insights with respect to research trends. In addition, a lot of important sentiment analysis datasets are summarized in this survey.

Bankruptcy prediction has long been a significant issue in finance and management science, which attracts the attention of researchers and practitioners. With the great development of modern information technology, it has evolved into using machine learning or deep learning algorithms to do the prediction, from the initial analysis of financial statements. In this paper, we will review the machine learning or deep learning models used in bankruptcy prediction, including the classical machine learning models such as Multivariant Discriminant Analysis (MDA), Logistic Regression (LR), Ensemble method, Neural Networks (NN) and Support Vector Machines (SVM), and major deep learning methods such as Deep Belief Network (DBN) and Convolutional Neural Network (CNN). In each model, the specific process of experiment and characteristics will be summarized through analyzing some typical articles. Finally, possible innovative changes of bankruptcy prediction and its future trends will be discussed.

Random forest is an ensemble learning method for classification, regression and other tasks that operate by constructing a multiple decision trees using training data and majority of the class will be consider as output. Out-of-Bag (OOB) takes the samples from the training set with replacement. In random forests, if you choose oob to true then there is no need for a separate test set to validate the model. It is estimated internally when the forest is built on training data, and each tree is tested on one-third of the samples not used in building that tree. Out of bag estimate an internal estimate of a random forest as it is being constructed. In this paper we propose two approach to implement outlier prediction by applying random forest classifier and LSTM model with experiments.

Web shell is one of the most common network attack methods, and traditional detection methods may not detect complex and flexible variants of web shell attacks. In this paper, we present a comprehensive detection system that can detect both PHP and JSP web shells. After file classification, we use different feature extraction methods, i.e. AST for PHP files and bytecode for JSP files. We present a detection model based on text processing methods including TF-IDF and Word2vec algorithms. We combine different kinds of machine learning algorithms and perform a comprehensively controlled experiment. After the experiment and evaluation, we choose the detection machine learning model of the best performance, which can achieve a high detection accuracy above 98%.

With the development of information technology, business intelligence and analytics (BI & A) has been applied in more and more fields. On the one hand, emerging application scenarios continue to appear; on the other hand, a huge number of multi-source heterogeneous data can be stored, greatly increasing the demand for data fusion, which requires a broader meaning. Traditional information fusion is mainly used in engineering fields. Based on the WSR systems methodology and the relative research, a new data fusion research framework of BI & A mainly used in economy, finance and management was constructed, which includes three fusion levels: data level, information level and knowledge level. At the data level, data can be selected from three dimensions to extract and construct understandable feature information. At the information level, multiple methods combined with domain knowledge are integrated to construct models and mine knowledge. At the knowledge level, low-level knowledge, expert opinions, decision-makers’ experience and other factors are fused to infer and summarize deeper knowledge and provide support for decision-making. The proposed data fusion framework of BI & A shows the important role in the whole business intelligence process, and some applications of data fusion in economy, finance and management were listed in the paper.

In a present-day world there are different types of attacks being launched on computing devices. World is experiencing more and more cyber-attacks and types of attacks are also increasing. For example, an IoT device in a home network can act as a botnet attacking other devices or there could be man in the middle attack. As time goes by more and more devices are being connected within any given network. All these devices will be vulnerable to attacks if any one of the devices is compromised within the network. This complicates Intrusion Detection in any given network. Manual detection and intervention is nearly impossible. Hence it is quintessential to detect different types of attacks with more confidence with less computation complexity and time. A lot of research work has already been done in this area where the attacks have been studied separately. In this paper we focus on detecting intrusions including IoT botnet attacks and other types of network attacks. To achieve this, we build a multiclass classification using supervised learning models along with the dimensionality reduction technique. Numerous studies on ML-based IDS have been using KDD or the upgraded versions of KDD dataset. In this study we used a new dataset, IoT network Intrusion Detection dataset.

Deep learning-based sentiment analysis (SA) methods have drawn more attention in recent years, which calls for more precise word embedding methods. This article proposes SentiVec, a kernel optimization function system for sentiment word embedding, which is based on two phases. The first phase is a supervised learning method, and the second phase consists of two unsupervised updating models, object-word-to-surrounding-words reward model (O2SR) and context-to-object-word reward model (C2OR). SentiVec is aimed at: 1) integrating the statistical information and sentiment orientation into sentiment word vectors and 2) propagating and updating the semantic information to all the word representations in a corpus. Extensive experimental results show that the optimal sentiment vectors successfully extract the features in terms of semantic and sentiment information, which makes it outperform the baseline methods on word similarity, word analogy, and SA tasks.

Under partial shading conditions, photovoltaic (PV) modules in a solar array experience varying irradiance. A Global Maximum (GM) and multiple Local Maximums (LMs) can originate on the Power-Voltage (P-V) curve under such non-uniform irradiance conditions. There are many maximum power point tracking (MPPT) algorithms developed to obtain maximum power. In the discrete environment of sampling, limited samples might be available to predict the true value of peak and as the change of environment is dynamic, limited time is available to make the prediction of the peak. Machine learning is used to overcome the two problems mentioned above as the MPPT algorithms are not robust enough. Various techniques are used to classify PV curve type and predict MPP. In this paper we process an algorithm to categorize the PV curves in best way and machine learning techniques are used to predict the range within which MPP is present.

Concepts have been adopted in concept-cognitive learning (CCL) and conceptual clustering for concept classification and concept discovery. However, the standard CCL algorithms are incapable of tackling continuous data directly, and some standard conceptual clustering methods mainly focus on the attribute information, ignoring the object information that is also important to improve clustering analysis and concept classification ability. Therefore, in this article, we present a novel concept learning method, called the fuzzy-based concept learning model (FCLM), to address these two issues by exploiting concept hierarchical relations in concept lattices. More specifically, we first show some new related notions for FCLM based on a regular fuzzy formal decision context; among these notions, the object-oriented and attribute-oriented fuzzy concept similarities are used to achieve the concept similarity measure in concept lattices. Moreover, a novel fuzzy concept learning framework is designed, and its corresponding learning algorithms are developed. Finally, we conduct some experiments on various real-world datasets to demonstrate that the proposed method can achieve the state-of-the-art classification performance among similarity-based learning methods. In addition, we further verify the effectiveness of our method in concept discovery on the MNIST dataset.

Stacking in machine learning allows multiple classification or regression algorithms to work together with a goal to enhance performance. To understand the risky properties of malware contamination in a system, it is important to accurately classify malware type first. Malware classification is the procedure of labeling the families of malware. In this paper, we automate stacking with 7 machine learning algorithms and 3 boosting algorithms. The experimental results show a 99.2% accuracy is achieved from a multilayer perceptron network with AdaBoost classifier, which outperforms other models on the malware API call dataset.

This paper develops a multi-task learning framework that attempts to incorporate the image structure knowledge to assist image inpainting, which is not well explored in previous works. The primary idea is to train a shared generator to simultaneously complete the corrupted image and corresponding structures --- edge and gradient, thus implicitly encouraging the generator to exploit relevant structure knowledge while inpainting. In the meantime, we also introduce a structure embedding scheme to explicitly embed the learned structure features into the inpainting process, thus to provide possible preconditions for image completion. Specifically, a novel pyramid structure loss is proposed to supervise structure learning and embedding. Moreover, an attention mechanism is developed to further exploit the recurrent structures and patterns in the image to refine the generated structures and contents. Through multi-task learning, structure embedding besides with attention, our framework takes advantage of the structure knowledge and outperforms several state-of-the-art methods on benchmark datasets quantitatively and qualitatively., Comment: Accepted by AAAI 2020

Nowadays, with the rapid development of network, the combination of online courses and traditional classrooms has become the trend of the times. In the traditional classroom, there are problems such as less teaching information, monotonous teaching methods, and resources cannot be shared. This paper is aimed at mechanical design, manufacturing and automation, focused on undergraduate teaching, taking the specialty course "Mechanical Manufacturing Technology" of Tongji University as an example, exploring an online video course based on "learning-oriented, teaching-imparting-new, short-to-long". The new teaching mode fully meets the students' needs for knowledge of practical engineering.

Relation Extraction systems train an extractor by aligning relation instances in Knowledge Base with a large amount of labeled corpora. Since the labeled datasets are very expensive, Distant Supervision Relation Extraction (DSRE) utilizes rough corpus annotated with Knowledge Graph to reduce the cost of acquisition. Nevertheless, the data noise problem limits the performance of the DSRE. Dependency trees can be used to filter the wrong-labeled instances in the distant supervision bag. However, existing dependency tree relation extraction strategies are all based on manually-set paths between the subject and object entities, and suffer from the problem of pruning the trees too aggressively or too insufficiently. To circumvent the shortcomings, in this paper, we propose a novel DSRE framework A2DSRE, based on the Adaptive dependency-path and Additional KG supervision. To obtain the dependency paths related to entity relations adaptively, we introduce an advanced graph neural network-GeniePath into DSRE, which assigns higher weights to those direct neighbor words that contribute more to relation prediction through breadth exploration, and conducts depth exploration to determine the correlation between relations and high-order neighbors. In this way, the irrelevant nodes are pruned while the relevant nodes are kept, our method can obtain more appropriate paths associated with relations. At the same time, to further reduce the noises in the data, we incorporate additional supervision information from the knowledge graph by retracting the margin between the representation of the bag and the pre-training knowledge graph embedding. Extensive numerical experiments validate the effectiveness of our new method.

Feature selection, as a fundamental component of building robust models, plays an important role in many machine learning and data mining tasks. Recently, with the development of sparsity research, both theoretical and empirical studies have suggested that the sparsity is one of the intrinsic properties of real world data and sparsity regularization has been applied into feature selection models successfully. In view of the remarkable performance of non-convex regularization, in this paper, we propose a novel non-convex yet Lipschitz continuous sparsity regularization term, named MCP $$^2$$ , and apply it into feature selection. To solve the resulting non-convex model, a new algorithm in the framework of the ConCave–Convex Procedure is given at the same time. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed method.

Online travel has developed dramatically during the past three years in China. This results in a large amount of unstructured data like tourism reviews from which it is hard to extract useful knowledge. In this paper, a DWWP system consisting of domain-specific new words detection (DW) and word propagation (WP) is presented. DW deals with the negligence of user-invented new words and converted sentiment words by means of AMI (Assembled Mutual Information). Inspired by social networks, the new method WP incorporates manually calibrated sentiment scores, semantic and statistical similarity information, which improves the quality of sentiment lexicon in comparison with existing data-driven methods. Experimental results show that DWWP improves seventeen percentage points compared with graph propagation and four percentage points compared with label propagation in terms of accuracy on Dataset I and Dataset II, respectively.

Online travel and online travel culture developed fast in China recently years while useful knowledge still hidden under a large number of tourism reviews. Therefore, we need effective sentiment analysis methods to mine useful knowledge which can help tourism websites make decisions and improve their travel products. Some data-driven sentiment lexicons have poor performance on sentiment polarity classification due to lack of semantic information. Thus, we propose an effective and more proper data-driven sentiment lexicon construction method incorporating manually labeled sentiment scores, semantic similarity information that is introduced by machine learning method word2vec. Experimental results demonstrate that our method improves the performance of tourism sentiment analysis significantly.

Species identification through DNA barcoding or metabarcoding has become a key approach for biodiversity evaluation and ecological studies. However, the rapid accumulation of barcoding data has created some difficulties: for instance, global enquiries to a large reference library can take a very long time. We here devise a two-step searching strategy to speed identification procedures of such queries. This firstly uses a Hidden Markov Model (HMM) algorithm to narrow the searching scope to genus level and then determines the corresponding species using minimum genetic distance. Moreover, using a fuzzy membership function, our approach also estimates the credibility of assignment results for each query. To perform this task, we developed a new software pipeline, FuzzyID2, using Python and C++. Performance of the new method was assessed using eight empirical data sets ranging from 70 to 234,535 barcodes. Five data sets (four animal, one plant) deployed the conventional barcode approach, one used metabarcodes, and two were eDNA-based. The results showed mean accuracies of generic and species identification of 98.60% (with a minimum of 95.00% and a maximum of 100.00%) and 94.17% (with a range of 84.40%-100.00%), respectively. Tests with simulated NGS sequences based on realistic eDNA and metabarcode data demonstrated that FuzzyID2 achieved a significantly higher identification success rate than the commonly used Blast method, and the TIPP method tends to find many fewer species than either FuzztID2 or Blast. Furthermore, data sets with tens of thousands of barcodes need only a few seconds for each query assignment using FuzzyID2. Our approach provides an efficient and accurate species identification protocol for biodiversity-related projects with large DNA sequence data sets.

On September 5, 2015, the State Council of Chinese Government, China’s cabinet formally announced its Action Framework for Promoting Big Data ( www.gov.cn , 2015). This is the milestone for China to catch up the global wave of big data. Since 2012 big data became a hot issue for scientific communities as well as the governments of many countries (Lazer et al. in Science 343:1203–1205, 2014; Einav et al. in Science 345:715, 2014; Cate in Science 346:818, 2014; Khoury and Ioannidis in Science 346:1054–1055, 2014). At the 2013 G8 Summit, the leaders of Canada, France, Germany, Italy, Japan, Russia, U.S.A. and United Kingdom agreed on an “open government plan” ( www.gov.uk/government/publications/open-data-charter/g8-open-data-charter-and-technical-annex , 2013). China’s framework, however, mainly emphasizes the integration of all trans-departmental data and establishes a number of government-driven national big data platforms so as to provide big data services to research, public and enterprises. The framework not only demonstrates a strong commitment of the Chinese government on big data, but also covers a wide range of governmental branches, enterprises and institutions far more than that of other countries. In addition, the framework shows an interpretation of big data that differs from other countries. If its objective is achieved, China would become a strong “big data country”.

Learning from label proportions is a new kind of learning problem which has drawn much attention in recent years. Different from the well-known supervised learning, it considers instances in bags and uses the label proportion of each bag instead of instance. As obtaining the instance label is not always feasible, it has been widely used in areas like modeling voting behaviors and spam filtering. However, learning from label proportions still suffers great challenges due to the inference of noise, the improper partition of bags and so on. In this paper, we propose a novel learning from label proportions method based on pinball loss, called “pSVM-pin”, to address the above issues. The pinball loss is introduced to generate an effective classifier in order to eliminate the impact of noise. Experimental results prove the precision of pSVM-pin compared with competing methods.

Malicious domain detection is one of the most effective approaches applied in detecting Advanced Persistent Threat (APT), the most sophisticated and stealthy threat to modern network. Domain name analysis provides security experts with insights to identify the Command and Control (C&C) communications in APT attacks. In this paper, we propose a machine learning based methodology to detect malware domain names by using Extreme Learning Machine (ELM). ELM is a modern neural network with high accuracy and fast learning speed. We apply ELM to classify domain names based on features extracted from multiple resources. Our experiment reveals the introduced detection method is able to perform high detection rate and accuracy (of more than 95%). The fast learning speed of our ELM based approach is also demonstrated by a comparative experiment. Hence, we believe our method using ELM is both effective and efficient to identify malicious domains and therefore enhance the current detection mechanism of APT attacks.

A robust and sparse multi-class approach for Multi-Class classification is proposed.The proposed method is based on Ramp loss K-Support Vector Classification-Regression.The CCCP procedure is used to solve a non-differentiable non-convex optimization problem.ADMM is adopted to make our model well-adapted for the large-scale setting.The results of Ramp-KSVCR show superior generalization power and low computational cost. Network intrusion detection problem is an ongoing challenging research area because of a huge number of traffic volumes, extremely imbalanced data sets, multi-class of attacks, constantly changing the nature of new attacks and the attackers methods. Since the traditional network protection methods fail to adequately protect the computer networks, the need for some sophisticated methodologies has been felt. In this paper, we develop a precise, sparse and robust methodology for multi-class intrusion detection problem based on the Ramp Loss K-Support Vector Classification-Regression, named Ramp-KSVCR. The main objectives of this research are to address the following issues; 1) Highly imbalanced and skewed attacks distribution; hence, we utilized the K-SVCR model as a core of our model; 2) Sensitivity of SVM and its extensions to the presence of noises and outliers in the training sets, to cope with this problem, Ramp loss function is implemented to our model; 3) and since the proposed Ramp-KSVCR model is a non-differentiable non-convex optimization problem, we took ConcaveConvex Procedure (CCCP) to solve this model. Furthermore, we introduced Alternating Direction Method of Multipliers (ADMM) procedure to make our model well-adapted to be applicable in the large-scale setting and to reduce the training time. The performance of the proposed method has been evaluated by some artificial data and also by conducting some experiments with the NSL-KDD data set and UNSW-NB15 as a recently published intrusion detection data set. Experimental results not only demonstrate the superiority of the proposed method over the traditional approaches tested against it in terms of generalization power and sparsity but also saving a considerable amount of computational time.

With the rapid development of the Computational Fluid Dynamics (CFD), Accurate computing hypersonic heating is in a high demand and still remains unresolved. In this paper, we propose a new scheme called AUSMPWM2 for hypersonic heating predictions. Based on systematic theoretical analyses, this new scheme improves the AUSMPWM scheme’s construction of the pressure flux for low speeds. Also, it constructs the mass flux as AUSMPWM to be robust against the shock anomaly and the unphysical expansion shock. Series of numerical experiments show that AUSMPWM2 is robust against the shock anomaly and avoids the expansion shock’s appearance. Also, it is capable of obtaining physical solutions at low speeds. Moreover, it is with a much higher level of accuracy in hypersonic heating predictions. These properties suggest that AUSMPWM2 is promising to be widely used to accurately and efficiently simulate both simple and complex flows.

Learning a classifier from groups of unlabeled data, only knowing, for each group, the proportions of data with particular labels, is an important branch of classification tasks that are conceivable in many practical applications. In this paper, we proposed a novel solution for the problem of learning with label proportions (LLP) based on nonparallel support vector machines, termed as proportion-NPSVM, which can improve the classifiers to be a pair of nonparallel classification hyperplanes. The unique property of our method is that it only needs to solve a pair of smaller quadratic programming problems. Moreover, it can efficiently incorporate the known group label proportions with the latent unknown observation labels into one optimization model under a large-margin framework. Compared to the existing approaches, there are several advantages shown as follows: 1) it does not need to make restrictive assumptions on the training data; 2) nonparallel classifiers can be achieved without computing the large inverse matrices; 3) the optimization model can be effectively solved by using the alternative strategy with SMO technique or SOR method; 4) proportion-NPSVM has better generalization ability. Sufficient experimental results on both binary-classes and multi-classes data sets show the efficiency of our proposed method in classification accuracy, which prove the state-of-the-art method for LLP problems compared with competing algorithms.

Sentiment analysis in tourism domain has drawn much attention in past few years, which calls for more precise sentiment word embedding method. The article proposes a kernel optimization function for sentiment word embedding. And the method aims at integrating the semantic information, statistics information and sentiment information and maintains the similarity between sentiment words in terms of sentiment orientation. The experiment result shows that the optimal sentiment vectors successfully extract the features in terms of sentiment information and the difference between concretization and abstraction of a sentiment words.

Large-scale linear classification is widely used in many areas. Although SVM-based models for ordinal regression problem are proven to be powerful techniques, the performance with nonlinear kernels are often suffering from time consuming. Recently, linear SVC not only is shown to obtain competitive performance in most of the cases, but also it is considerably fast during the process of training and testing. However, few studies focused on linear SVM-based ordinal regression models. In this paper, we propose a new approach, called linear Nonparallel Support Vector Ordinal Regression (NPSVOR), which can deal with large-scale problems. An efficient algorithm based on Alternating Direction Method of Multipliers (ADMM) is designed to solve the proposed model. Our experiments are performed on large document data sets to demonstrate the effectiveness of the proposed method.

Chinese segmentation has attracted amounts of attention in natural language processing in recent years and is the basis of web text mining. The article improved statistics-based method EMI, then we use improved approach to detect new words in tourism field. The result demonstrates that our method can detect new words significantly, especially in detecting proper nouns and sentiment words which will be helpful in subsequent tasks such as sentiment analysis and word embedding. In additional, this paper analyze parameters which are influential on the effects of new words detection. At last, the article discussed possible application of new word detection in sentiment analysis.

Summary 1.Species identification via DNA barcodes has recently become an important and routine task in many biodiversity projects using DNA sequence data. 2.Here, we present BarcodingR, an integrated software package that provides a comprehensive implementation of species identification methods, including artificial intelligence, fuzzy-set, Bayesian, and kmer-based methods, that are not readily available in other packages. 3.BarcodingR additionally provides new functions for barcode evaluation, barcoding gap analysis, delimitation comparison analysis, species membership analysis, and consensus identification. 4.Comparison with other barcoding methods using eleven empirical datasets indicates that on average, FZKMER (implemented in BarcodingR) and one extant barcoding method BRONX outperform all other methods examined in this study. Two other methods, BP and FZ (both implemented in BarcodingR), present similar performance as SVM and BLOG respectively, and all display better performance than that of Jrip. 5.The software of BarcodingR is open source under GNU General Public License, and freely available for all major operating systems. This article is protected by copyright. All rights reserved.

We are in the era of information explosion, and everyday a large amount of data are generated from all kinds of sources such as social media, biology, online purchase, etc. It is crucial that we acquire data mining tools to efficiently and effectively analyze the data and gather useful information. There are various aspects of data mining approaches such as clustering, outlier detection and dimension reduction. In this paper we propose an approach to selecting dimensions based on the information of outliers on different dimensions, along with the information of clusters. We also conduct experiments to demonstrate the performance of our algorithm.

Learning from label proportions (LLP) is a new kind of learning problem which has attracted wide interest in machine learning. Different from the well-known supervised learning, the training data of LLP is in the form of bags and only the proportion of each class in each bag is available. Actually, many modern applications can be successfully abstracted to this problem such as modeling voting behaviors and spam filtering. However, time-consuming training is still a challenge for LLP, which becomes a bottleneck especially when addressing large bags and bag sizes. In this paper, we propose a fast algorithm called multi-class learning from label proportions by extreme learning machine (LLP-ELM), which takes advantage of an extreme learning machine with fast learning speed to solve multi-class learning from label proportions. Firstly, we reshape the hidden layer output matrix and the training data target matrix of an extreme learning machine to adapt to the proportion information instead of the real labels. Secondly, a robust loss function with a regularization term is formulated and two efficient solutions are provided to different cases. Finally, various experiments demonstrate the significant speed-up of the proposed model with better accuracies on different datasets compared with several state-of-the-art methods.

Semantic Segmentation is a computer vision task for predicting the pixel labels corresponding to its belonging region or enclosing region area. It is an important part in many CV tasks and plays a significant role in machine learning. Semantic segmentation is aim at understanding special object class in the scene. In the paper, we will give a survey of Semantic Segmentation. At first, we make a brief introduction of Semantic Segmentation, introducing the wide use of semantic segmentation. Its range is from scene understanding, humanmachine interaction, computational photography, image search engine, predicting for the relationships of multiple objects mutual support in autonomous driving area. Next, we divide the Semantic Segmentation methods into two classes by the input modalities number. We make a big survey for different methods based on different structure, and show the reasons why these methods were introduced and how did they perform on the dataset. We figure out their contributions and significance. Then, we compare the basic dataset used in each method.

Compared with traditional methods, word em-bedding is an efficient language representation that can learn syntax and semantics by using neural networks. As the result, more and more promising experiments in natural language processing (NLP) get the state-of-the-art results by introducing word embedding. In principle, embedding representation learning embeds words to a low-dimensional vector space, there-fore vectors support initialization of NLP tasks such as text classification, sentiment analysis, language understanding, etc. However, polysemy is very common in many languages, which causes word ambiguation, further influences the accuracy of the system. Additionally, language models based on distributed hypotheses mostly focused on word properties rather than morphology were our primary focus. This leads to unreasonable performance in different evaluations. At the same time, word embedding learning and measuring are two vital components of word representation. In this paper, we overviewed many language models including single sense and multiple sense word embedding, and many evaluated approaches including intrinsic and extrinsic evaluation. We found that there are obvious gaps between vectors and manual annotations in word similarity evaluation, and language models that achieved good performance in intrinsic evaluations could not produce similar results in extrinsic evaluations. To the best of our knowledge, there is no universal language model and embedding learning method for most NLP task, and each evaluations also hidden natural defects compared to human knowledge. More evaluated datasets are also investigated such as datasets used in intrinsic and extrinsic evaluations. We believe that an improved evaluation dataset and a more rational evaluation method would benefit from this overview.

The novel node coupling clustering methods for link prediction are proposed.A new node coupling degree metric is proposed.The node coupling information and clustering information are used.Experimental evaluation about the effectiveness of our methods is presented. Due to the potential important information in real world networks, link prediction has become an interesting focus of different branches of science. Nevertheless, in "big data" era, link prediction faces significant challenges, such as how to predict the massive data efficiently and accurately. In this paper, we propose two novel node-coupling clustering approaches and their extensions for link prediction, which combine the coupling degrees of the common neighbor nodes of a predicted node-pair with cluster geometries of nodes. We then present an experimental evaluation to compare the prediction accuracy and effectiveness between our approaches and the representative existing methods on two synthetic datasets and six real world datasets. The experimental results show our approaches outperform the existing methods.

Sentiment analysis, which refers to the task of detecting whether a textual item (e.g., a product review and a blog post) expresses a positive or negative opinion in general or about a given entity (e.g., a product, person, or policy), has received increasing attention in recent years. It serves as an important role in natural language processing. User generated content, like tourism reviews, developed dramatically during the past years, generating a large amount of unstructured data from which it is hard to obtain useful information. Due to the changes in textual order, sequence length and complicated logic, it is still a challenging task to predict the exact sentiment polarities of the user reviews, especially for fine-grained sentiment classification. In this paper, we first propose sentiment padding, a novel padding method compared with zero padding, making the input data sample of a consistent size and improving the proportion of sentiment information in each review. Inspired by the most recent studies with respect to neural networks, we propose deep learning based sentiment analysis models named lexicon integrated two-channel CNN-LSTM family models, combining CNN and LSTM/BiLSTM branches in a parallel manner. Experiments on several challenging datasets, like Stanford Sentiment Treebank, demonstrate that the proposed method outperforms many baseline methods.

Corporate bankruptcy prediction is an interesting and important research topic that can be conceived in many practical applications. Recently, machine learning based methods have been widely proposed to solve the problem of bankruptcy prediction. However, the existing models do not consider that large amounts of instance-level labeled training data are hard to be obtained in practice. In this paper, we propose to address bankruptcy prediction problem from the perspective of learning with label proportions, where the unlabeled training data are provided in different bags and only giving the bag-level proportion of instances belonging to a particular class. Then, we contribute two novel prediction methods, termed as Bagged-pSVM and Boosted-pSVM, based on proportion support vector machines and ensemble strategies including bagging and boosting. The proposed methods can not only explicitly model the unknown instance-level labels and the known label proportions under a large-margin framework, but also improve the performance through introducing ensemble learning strategies. Extensive experiments on the benchmark datasets demonstrate their efficiency and superiority on solving the problem of bankruptcy prediction.

Word representation is one foundational research in natural language processing which full of challenges compared to other fields such as image and speech processing. It embeds words to a dense low-dimensional vector space and is able to learn syntax and semantics at the same time. But this representation only get one single vector for a word no matter it is polysemy or not. In order to solve this problem, sense information are added in the multiple sense language models to learn alternative vectors for each single word. However, as the most popular measuring method in single sense language models, word similarity did not get the same performance in multiple situation, because word similarity based on cosine distance doesn’t match annotated similarity scores. In this paper, we analyzed similarity algorithms and found there is obvious gap between cosine distance and benchmark datasets, because the negative internal in cosine space does not correspond to manual scores space and cosine similarity did not cover semantic relatedness contained in datasets. Based on this, we proposed a new similarity methods based on mean square error and the experiments showed that our new evaluation algorithm provided a better method for word vector similarity evaluation.

In large-scale learning problem, the scalability of learning algorithms is usually the key factor affecting the algorithm practical performance, which is determined by both the time complexity of the learning algorithms and the amount of supervision information (i.e., labeled data). Learning with label proportions (LLP) is a new kind of machine learning problem which has drawn much attention in recent years. Different from the well-known supervised learning, LLP can estimate a classifier from groups of weakly labeled data, where only the positive/negative class proportions of each group are known. Due to its weak requirements for the input data, LLP presents a variety of real-world applications in almost all the fields involving anonymous data, like computer vision, fraud detection and spam filtering. However, even through the required labeled data is of a very small amount, LLP still suffers from the long execution time a lot due to the high time complexity of the learning algorithm itself. In this paper, we propose a very fast learning method based on inversing output scaling process and extreme learning machine, namely Inverse Extreme Learning Machine (IELM), to address the above issues. IELM can speed up the training process by order of magnitudes for large datasets, while achieving highly competitive classification accuracy with the existing methods at the same time. Extensive experiments demonstrate the significant speedup of the proposed method. We also demonstrate the feasibility of IELM with a case study in real-world setting: modeling image attributes based on ImageNet Object Attributes dataset.

In online advertisement industry, it is important to predict potentially profitable users who will click target ads (i.e., Behavioral targeting). The task selects the potential users that are likely to click the ads by analyzing user's clicking/web browsing information and displaying the most relevant ads to them. This paper proposes four multiple criteria mathematical programming models for advertisement clicking problems. First two are multi-criteria linear regression (MCLR) and kernel-based multiple criteria regression (KMCR) algorithms for click-through rate (CTR) prediction. The second two are multi-criteria linear programming (MCLP) and kernel-based multiple criteria programming (KMCP) algorithms, which are used to predict ads clicking events, such as identifying clicked ads in a set of ads. Using the experimental datasets from KDD Cup 2012, the paper first conducts a comparison of the proposed MCLR and KMCR with the methods of support vector regression (SVR) and logistic regression (LR), which shows that both MCLR and KMCR are good alternatives. Then the paper further studies the performance between the proposed MCLP and KMCP algorithms with known algorithms, including support vector machines (SVM), LR, radial basis function network (RBFN), k-nearest neighbor algorithm (KNN) and Naive Bayes (NB) in both prediction and selection processes. The studies show that the MCLP and KMCP models have better performance stability and can be used to effectively handle behavioral targeting application for online advertisement problems.