Search

This paper examines the work practices involved in making data legible to machines and machine output legible to humans. The study is based on ethnographic research of a team of art experts at DNArt – a data classification system that features a growing database of art images, a classification scheme, a similarity matching algorithm, and a website that together serve as a consumer judgment device in an emerging online market for art. I analyze interactions from meeting observations, interviews, documentation, and online interaction data to show how non-technical art experts explain and repair sociotechnical breakdowns – when their expectations for similarity between art images and artists differ from the similarity relations produced by the algorithm. By repairing breakdowns, the art experts construct the algorithm anew, as a legitimate revealer of similarity in art. In doing so, the team's repair work is folded back into the black box of the algorithm, rendering it invisible and unacknowledged, sometimes even by the experts themselves. [ABSTRACT FROM AUTHOR]

Applying Machine Learning to a problem which involves medical, financial, or other types of sensitive data needs careful attention in order to maintaining data privacy and security. This paper presents a model for privacy preserving classification and demonstrated that, by using a decision tree classifier, it is possible to perform a privacy preserving classification operation on an encrypted data residing on an untrusted server using the technique of Fully Homomorphic Encryption. First, the paper presented a model for the design and implementation of privacy preserving decision tree classifier over encrypted data. Also, Fully Homomorphic Encryption technique was used to secretly carry out classification on ciphertext using decision tree model built out of confidential medical data. The classifier was implemented using the SEAL homomorphic library and evaluation was done using encrypted medical datasets. The experimental results demonstrated high accuracy of the ciphertext classifier (when compared to the plaintext data equivalent) and efficiency (compared to other classifier on similar tasks). It takes less than 5 seconds (depending on the depth) to perform classification over an encrypted hepatitis feature vector dataset. [ABSTRACT FROM AUTHOR]

Big data analytics uncovers hidden patterns through classification, prediction and reinforcement of big datasets. In these datasets, some features have a negligible connection with other features and some may be insignificant as their presence does not impact the results of big data analytics. The algorithms of big data analytics generate better classification models when supplied with a dataset consisting of relevant, important and informative features. These features can be classified as important and unimportant. For the selection of important features, different filtrations techniques are used. These techniques filter features on different basis like information gain, information dispersion, Gini index, etc. and have a few drawbacks reviewed in this paper. The first contribution of this paper is to propose a new feature selection technique named "Relevance-diversity algorithm" for selecting important features based on two measures i.e. relevance and diversity for optimizing features as low as possible and reducing the search time used in feature selection. The second contribution of the paper is that it proposes a new supervised classification algorithm based on Naive Bayes classification. The assumption of naive i.e. feature independence is discarded from the algorithm of Naive Bayes classification. The features are considered to be dependent on each other and their combined impact on the class value is evaluated. The newly proposed classification algorithm is then applied to the features selected through the relevance-diversity based feature selection technique. The datasets of Weather, Tic-Tac-Toe, Lenses, Balance-scale and CarEval are used for the evaluation of both the techniques. The results of the proposed feature selection method are compared with the existing methods and the results of Modified-Bayes are compared with the existing Naive Bayes algorithm. Analysis revealed that the proposed method performed better in terms of the number of features, accuracy and time complexity. [ABSTRACT FROM AUTHOR]

Because there are lots of typical applications and urgent needs, the research on the efficient classification learning about accumulated big data in nonstationary environments has become one of the hot topics in the field of data mining recently. The LearnNSE algorithm is an important research result in this field. For the long-term accumulated big data, the LearnNSE-Pruned-Age, a pruning version of LearnNSE, was given, which has received widespread attentions. However, it is found that the pruning mechanism of the LearnNSE-Pruned-Age algorithm is not perfect, which lost the core ability of the LearnNSE algorithm to reuse the learned classification knowledge. Therefore, the ensemble mechanism of LearnNSE is adjusted in this paper, and a novel ensemble mechanism is designed. The new mechanism uses the integration of the latest base-classifiers to track the changes of the data generation environment, and then selects the old base-classifiers that contribute to the current classification for forward supplementary integration. On this basis, a new pruned algorithm named FLearnNSE-Pruned-Age is proposed. The experiment results show that the FLearnNSE-Pruned-Age algorithm has the ability to reuse the learned classification knowledge and it can achieve the very close classification accuracy compared to LearnNSE, even better in some scenes. In addition, it improves the efficiency of ensemble learning and is suitable for the fast classification learning of accumulated big data. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Feature selection has been an important issue in machine learning and data mining, and is unavoidable when confronting with high‐dimensional data. With the advent of multilabel (ML) datasets and their vast applications, feature selection methods have been developed for dimensionality reduction and improvement of the classification performance. In this work, we provide a comprehensive review of the existing multilabel feature selection (ML‐FS) methods, and categorize these methods based on different perspectives. As feature selection and data classification are closely related to each other, we provide a review on ML learning algorithms as well. Also, to facilitate research in this field, a section is provided for setup and benchmarking that presents evaluation measures, standard datasets, and existing software for ML data. At the end of this survey, we discuss some challenges and open problems in this field that can be pursued by researchers in future. WIREs Data Mining Knowl Discov 2018, 8:e1240. doi: 10.1002/widm.1240 This article is categorized under: Technologies > Data Preprocessing [ABSTRACT FROM AUTHOR]

The novelty in big data is rising day-by-day in such a way that the existing software tools face difficulty in supervision of big data. Furthermore, the rate of the imbalanced data in the huge datasets is a key constraint to the research industry. Thus, this paper proposes a novel technique for handling the big data using Spark framework. The proposed technique undergoes two steps for classifying the big data, which involves feature selection and classification, which is performed in the initial nodes of Spark architecture. The proposed optimization algorithm is named rider chaotic biography optimization (RCBO) algorithm, which is the integration of the rider optimization algorithm (ROA) and the standard chaotic biogeography-based optimisation (CBBO). The proposed RCBO deep-stacked auto-encoder using Spark framework effectively handles the big data for attaining effective big data classification. Here, the proposed RCBO is employed for selecting suitable features from the massive dataset. [ABSTRACT FROM AUTHOR]

The $$k$$ -NN classifier is a widely used classification algorithm. However, exhaustively searching the whole dataset for the nearest neighbors is prohibitive for large datasets because of the high computational cost involved. The paper proposes an efficient model for fast and accurate nearest neighbor classification. The model consists of a non-parametric cluster-based preprocessing algorithm that constructs a two-level speed-up data structure and algorithms that access this structure to perform the classification. Furthermore, the paper demonstrates how the proposed model can improve the performance on reduced sets built by various data reduction techniques. The proposed classification model was evaluated using eight real-life datasets and compared to known speed-up methods. The experimental results show that it is a fast and accurate classifier, and, in addition, it involves low pre-processing computational cost. [ABSTRACT FROM AUTHOR]

This paper proposes an evolutionary-based selective ensemble learning framework for solving classification problem. In the proposed ensemble learning framework, extreme learning machine (ELM) is selected as base learner and evolutionary algorithms are employed to optimize the weights of base learners in the ensemble. Then, some base learners, that their weights are larger than the threshold, are selected for making decision. The proposed ensemble learning framework is evaluated on 20 benchmark data sets from KEEL repository through four different evolutionary algorithms. Results show that the proposed evolutionary-based ensemble learning framework outperforms the simple voting based ensemble method in terms of classification performance. In four evolutionary optimization algorithms, PSOGA-based and DE-based weight optimization algorithms can effectively improve the classification accuracy and generalization ability. [ABSTRACT FROM AUTHOR]

The paper presents an algorithm of decision rules redefinition that is based on evaluation of the importance of elementary conditions occurring in induced rules. Standard and simplified (heuristic) indices of elementary condition importance evaluation are described. There is a comparison of the results obtained by both indices concerning classifiers quality and elementary condition rankings estimated by the indices. The efficiency of the proposed algorithm has been verified on 21 benchmark data sets. Moreover, an analysis of practical applications of the proposed methods for biomedical and medical data analysis is presented. The obtained results show that the redefinition reduces considerably a rule set needed to describe each decision class. Additionally, after the rule set redefinition negated elementary conditions may also occur in new rules. [ABSTRACT FROM AUTHOR]

Despite the rapid improvement in dental health over the last few decades, a significant portion of our population continue seek dental care every year. Estimates show that 13% of adults seek dental care for dental infection or fatigue within four years. The Social and individual burden of this disease can be reduced by its early detection. However, the symptoms of teeth infection in the early stages are not clear, hence, it would be relatively difficult to predict teeth infections based solely on human skills and experience. Big Data (BD) technologies have a great potential in transforming dental care, as they have revolutionized other industries. In addition to reducing cost, they could save millions of lives and improve patient outcomes. This paper proposes a novel integrated prediction model that extracts hidden knowledge from radiographic datasets containing a large volume of dental X-ray images and utilizes this knowledge to predict dental infections. Initially, preprocessing techniques using morphological skeleton and mean approach is applied to eliminate noise and enhance the images. Next, Multi Scale Segmented Region (MSR) approach, Watershed Approach (WA), Sobel edge Detection (SD), Histogram based Segmentation (HS), Trainable Segmentation (TS), Dual Clustering (DC), and Fuzzy C-Means clustering (FCM) are examined for image segmentation and feature extraction. Among these methods, MSR was selected for feature extraction since it outperformed other methods in terms of accuracy, specificity, precision, recall and F1-score. Then, a set of neural network classifiers are trained to identify patterns in the extracted optimized features and predict dental infections. For this purpose, we have examined Bacterial Optimized Recurrent Neural Networks (BORNN), Deep Learning Neural Networks (DANN), Genetic Optimized Neural Networks (GONN) and Adaptive Neural Networks Algorithm (ADNN). BORNN have shown maximum accuracy and Roc value (98.1% and 0.92 respectively), and minimum error values (MSE = 0.189, MAE = 0.143). The output of the proposed integrated prediction model is fed into a dental robot who proceeds with the treatment process with high accuracy and minimum delay. The proposed prediction model was implemented using a big data analytics tool called Apache SAMOA and experimental results showed its correctness and effectiveness. [ABSTRACT FROM AUTHOR]

An exponential rise has been observed in the data volume over the time when considering a real time environment. A phenomenal feature termed as 'Predictability' helps in predicting and portraying related data to the user according to their needs. Moreover, classification of Big Data is usually a tedious and lengthy task. The technique of MapReduce Framework performs the data processing that being paralleled by data distribution in small chunks through the clusters. This Map Reduce technique is being proposed which is employed to process heterogeneous data items. Few issues that are being targeted in the existing paper include associating climatologically and meteorological information with large variety of farming decisions. Using the well-known MapReduce framework the above issues and challenges can be resolved. The existing paper proposes empirical techniques of climate classification and prediction by adopting Co-EANFS (Co-Effective and Adaptive Neuro-Fuzzy System) approach for data handling. Furthermore, the paper examines association rule mining too, which is being implemented for examining the best crop production by relying upon the soil and weather condition. Lastly, a technique is proposed for managing various levels such as preprocessing, clustering, classification and prediction. First, the weather dataset is being collected which undergoes processing; thereafter the proposed model is implemented which results in formation of cluster data sets linked to each season. For evaluating the performance, accuracy predictions generated by Co-EANFS is used which being formulated with varying no: of inputs and variables. The proposed framework acquires least execution time. [ABSTRACT FROM AUTHOR]

Copyright of Radio Electronics, Computer Science, Control is the property of Zaporizhzhia National Technical University and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Multi-label problems arise in various domains, including automatic multimedia data categorization, and have generated significant interest in computer vision and machine learning community. However, existing methods do not adequately address two key challenges: exploiting correlations between labels and making up for the lack of labelled data or even missing labelled data. In this paper, we proposed to use a semi-supervised singular value decomposition (SVD) to handle these two challenges. The proposed model takes advantage of the nuclear norm regularization on the SVD to effectively capture the label correlations. Meanwhile, it introduces manifold regularization on mapping to capture the intrinsic structure among data, which provides a good way to reduce the required labelled data with improving the classification performance. Furthermore, we designed an efficient algorithm to solve the proposed model based on the alternating direction method of multipliers, and thus, it can efficiently deal with large-scale data sets. Experimental results for synthetic and real-world multimedia data sets demonstrate that the proposed method can exploit the label correlations and obtain promising and better label prediction results than the state-of-the-art methods. [ABSTRACT FROM PUBLISHER]

We propose a generalized Sparse Representation-based Classification (SRC) algorithm for open set recognition where not all classes presented during testing are known during training. The SRC algorithm uses class reconstruction errors for classification. As most of the discriminative information for open set recognition is hidden in the tail part of the matched and sum of non-matched reconstruction error distributions, we model the tail of those two error distributions using the statistical Extreme Value Theory (EVT). Then we simplify the open set recognition problem into a set of hypothesis testing problems. The confidence scores corresponding to the tail distributions of a novel test sample are then fused to determine its identity. The effectiveness of the proposed method is demonstrated using four publicly available image and object classification datasets and it is shown that this method can perform significantly better than many competitive open set recognition algorithms. [ABSTRACT FROM PUBLISHER]

Due to the insidious characteristics of network intrusion behaviors, developing an efficient intrusion detection system is still a big challenge, especially in the era of big data where the number of traffic and the dimension of each traffic feature are high. Because of the shortcomings of traditional common machine learning algorithms in network intrusion detection, such as insufficient accuracy, a network intrusion detection system based on LightGBM and autoencoder (AE) is proposed. The LightGBM-AE model proposed in this paper includes three steps: data preprocessing, feature selection, and classification. The LightGBM-AE model adopts the LightGBM algorithm for feature selection, and then uses an autoencoder for training and detection. When a set of data containing network intrusion behaviors are inputted into an autoencoder, there is a large reconstruction error between the original input data and the reconstructed data obtained by the autoencoder, which provides a basis for intrusion detection. According to the reconstruction error, an appropriate threshold is set to distinguish symmetrically between normal behavior and attack behavior. The experiment is carried out on the NSL-KDD dataset and implemented using Pytorch. In addition to autoencoder, variational autoencoder (VAE) and denoising autoencoder (DAE) are also used for intrusion detection and are compared with existing machine learning algorithms such as Decision Tree, Random Forest, KNN, GBDT, and XGBoost. The evaluation is carried out through classification evaluation indexes such as accuracy, precision, recall, F1-score. The experimental results show that the method can efficiently separate the attack behavior from normal behavior according to the reconstruction error. Compared with other methods, the effectiveness and superiority of this method are verified. [ABSTRACT FROM AUTHOR]

Aiming at the challenges faced by the existing big data-oriented computing framework in the study of extensible machine learning, this paper proposed a distributed naive Bayesian text classification method based on MapReduce and Apache Spark framework. This method explored the Bayesian network classifier by studying the adaptability of MapReduce and Apache Spark frameworks, and studied the existing computing framework for big data. First, it divided the training sample data set into m classes based on the naive Bayes text classification model. In the training phase, it used the output of the previous MapReduce as the input of the next MapReduce, and used four MapReduce jobs to derive the model. This design process made full use of the parallel advantages of MapReduce. Finally, when the classifier was tested, it obtained the value of the class label which the maximum value belonged. Experiments in the Newgroup's dataset show the proposed method achieves more than 99% of the results on all five types of news data sets, and is all higher than the comparison algorithms, which prove the accuracy of the method. [ABSTRACT FROM AUTHOR]

Selection of appropriate nearest neighbors greatly affects predictive accuracy of nearest neighbor classifier. Feature similarity is often used to decide the set of k nearest neighbors. Predictive accuracy of multi-label kNN could further be enhanced if in addition to the feature similarity, difference in feature values and dissimilarity of the instance labels are also taken into account to decide the set of k nearest neighbors. This paper deals with an algorithm called "ML-FLD" that not only takes into account features similarity of the instances, but also considers feature difference and label dissimilarity in order to decide the k nearest neighbors of a given unseen instance for the prediction of its labels. The algorithm when tested using well-known datasets and checked with the existing well known algorithms, provides better performance in terms of examplebased metrics such as hamming loss, ranking loss, one error, coverage, average precision, accuracy, F measure as well as label-based metrics like macro-averaged and micro-averaged F measure. [ABSTRACT FROM AUTHOR]

In the identification of network text information, the existing technology is difficult to accurately extract and classify text information with high propagation speed and high update speed. In order to solve this problem, the research combines the Naive Bayes algorithm with the feature two-dimensional information gain weighting method, uses the feature weighting method to optimize the Naive Bayes algorithm, and calculates the dimension of different documents and data categories through a new feature operation method. The data gain between them can improve its classification performance, and the classification models are compared and analyzed in the actual Chinese and English databases. The research results show that the classification accuracy rates of the IGDC-DWNB model in the Sogou database, 20-newsgroup database, Fudan database and Ruster21578 database are 0.89, 0.89, 0.93, and 0.88, respectively, which are higher than other classification models in the same environment. It can be seen that the model designed in the research has higher classification accuracy, stronger overall performance, and stronger reliability and robustness in practical applications, which can provide a new development idea for big data classification technology. [ABSTRACT FROM AUTHOR]

There is a crucial need to process patient's data immediately to make a sound decision rapidly; this data has a very large size and excessive features. Recently, many cloud-based IoT healthcare systems are proposed in the literature. However, there are still several challenges associated with the processing time and overall system efficiency concerning big healthcare data. This paper introduces a novel approach for processing healthcare data and predicts useful information with the support of the use of minimum computational cost. The main objective is to accept several types of data and improve accuracy and reduce the processing time. The proposed approach uses a hybrid algorithm which will consist of two phases. The first phase aims to minimize the number of features for big data by using the Whale Optimization Algorithm as a feature selection technique. After that, the second phase performs real-time data classification by using Naïve Bayes Classifier. The proposed approach is based on fog Computing for better business agility, better security, deeper insights with privacy, and reduced operation cost. The experimental results demonstrate that the proposed approach can reduce the number of datasets features, improve the accuracy and reduce the processing time. Accuracy enhanced by average rate: 3.6% (3.34 for Diabetes, 2.94 for Heart disease, 3.77 for Heart attack prediction, and 4.15 for Sonar). Besides, it enhances the processing speed by reducing the processing time by an average rate: 8.7% (28.96 for Diabetes, 1.07 for Heart disease, 3.31 for Heart attack prediction, and 1.4 for Sonar). [ABSTRACT FROM AUTHOR]

Relief-like algorithms have been widely used as feature selection to reduce the dimension of high-dimensional data which involves thousands of irrelevant variables because of their low computational cost and high accuracy. Classical Relief algorithms have not exactly shown the dynamic procedure that updates weight iteratively. This paper proposes an innovative feature weight estimation method, called dynamic representation and neighbor sparse reconstruction-based Relief (DRNSR-Relief). Similar to the classical Relief algorithms, the goal of DRNSR-Relief is to maximize the expected margin in the weighted feature space. A dynamic representation framework is introduced to show the dynamic relationship between the expected margin vector and the weight vector. To achieve better neighbor reconstruction, DRNSR-Relief decomposes a nonlinear problem into a set of locally linear ones through local hyperplane with l 1 regularization and then estimates feature weights in a large margin framework. With the help of gradient ascent method, we can guarantee the convergence of DRNSR-Relief. To demonstrate the validity and the effectiveness of our formulation for feature selection in supervised learning, we perform extensive experiments on synthetic and real-world datasets. Experimental results indicate that DRNSR-Relief is very promising. [ABSTRACT FROM AUTHOR]

User interfaces based on mid-air gesture recognition are expected to become popular in the near future due to the increasing diffusion of virtual, mixed reality applications and smart devices. The design of this kind of interfaces would be clearly helped by the availability of simple and effective methods to compare short 3D trajectories, allowing fast and accurate recognition of command gestures given a few examples. This approach, quite popular in 2D touch-based interfaces with the so-called “dollar” algorithm family, has not been deeply investigated for 3D mid-air gestures. In this paper, we explore several metrics that can be used for mid-air gesture comparison and present experimental tests performed to analyze their effectiveness on practical tasks. By adopting smart choices in gesture traces processing and comparing, it was possible to obtain very good results in the retrieval and recognition of simple command gestures, from complete or even partial hand trajectories. The approach was also extended in order to recognize gestures characterized by both hand and finger motions and tested on a recent benchmark, reaching state of the art performances. [ABSTRACT FROM AUTHOR]

Decision tree is one of the most popular classification methods because of its advantages of easy comprehension. However, the decision tree constructed by existed methods is usually too large and complicated. So, in some applications, the practicability is limited. In this paper, combining NOLCDT with IID5R algorithm, an improved hybrid classifier algorithm, HCS, is proposed. HCS algorithm consists of two phases: building initial decision tree and incremental learning. The initial decision tree is constructed according to the NOLCDT algorithm, and then the incremental learning is performed with IID5R. The NOLCDT algorithm selects the candidate attribute with the largest information gain and divides the node into two branches, which avoids generating too many branches. Thus, this prevents the decision tree is too complex. The NOLCDT algorithm also improves on the selection of the next node to be split, which computes the corresponding nodal splitting measure for all candidate splits, and always selects the node which has largest information gain from all candidate split nodes as the next split node, so that each split has the greatest information gain. In addition, based on ID5R, an improved algorithm IID5R is proposed to evaluate the quality of classification attributes and estimates a minimum number of steps for which these attributes are guaranteed such a selection. HCS takes advantage of the decision tree and the incremental learning method, which is easy to understand and suitable for incremental learning. The contrast experiment between the traditional decision tree algorithm and HCS algorithm with UCI data set is proposed; the experimental results show that HCS can solve the increment problem very well. The decision tree is simpler so that it is easy to understand, and so the incremental phase consumes less time. [ABSTRACT FROM AUTHOR]

A CART decision tree algorithm based on attribute weight is proposed in this paper because of the present problems of complex classification, poor accuracy, low efficiency, and severe memory consumption of CART decision. What is more, the algorithm is combined with the parallel computing model of MapReduce. Theory of attribute weights is used in the algorithm. A decision tree is built through the sum of weights, which is decided by the degree that the attributes affect a decision. Thus the accuracy of classification through decision tree is improved. Parallel sorting algorithms of CART decision tree for massive data is implemented through the MapReduce programming technology of cloud computing. All the results of theoretical analysis and experimental comparison show that it is very important to mark attributes by weights through MapReduce. Furthermore, the accuracy of the classification of large sample data sets is improved significantly, classification efficiency of decision tree is improved and the trained time is also significantly reduced. [ABSTRACT FROM AUTHOR]

Feature selection, both for supervised as well as for unsupervised classification is a relevant problem pursued by researchers for decades. There are multiple benchmark algorithms based on filter, wrapper and hybrid methods. These algorithms adopt different techniques which vary from traditional search-based techniques to more advanced nature inspired algorithm based techniques. In this paper, a hybrid feature selection algorithm using graph-based technique has been proposed. The proposed algorithm has used the concept of Feature Association Map (FAM) as an underlying foundation. It has used graph-theoretic principles of minimal vertex cover and maximal independent set to derive feature subset. This algorithm applies to both supervised and unsupervised classification. The performance of the proposed algorithm has been compared with several benchmark supervised and unsupervised feature selection algorithms and found to be better than them. Also, the proposed algorithm is less computationally expensive and hence has taken less execution time for the publicly available datasets used in the experiments, which include high-dimensional datasets. [ABSTRACT FROM AUTHOR]

Ensemble feature selection is one of the ensemble learning methods, where each classifier is trained or built by feature selection result. Ensemble feature selection is an effective way for dealing with high dimension and small sample data, such as microarray data. However, ensemble feature selection should achieve more accurate and stable classification performance. In this paper, we present a novel diversity measure based on information theory called Sum of Minimal Information Distance (SMID), which maximizes the relevance between feature subsets and class label as well as the diversity between feature subsets. Moreover, a novel ensemble feature selection framework satisfying this criterion is proposed. In this framework, features that have more mutual information with class label and more diversity between each other are retained. Different feature subsets are used to train base classifiers after being obtained by incremental search method, and then these classifiers are aggregated into a consensus classifier by majority voting. Comparing with three representative feature selection methods and five ensemble learning methods on ten microarray datasets, the experiment results show that the proposed method achieves better performance than the other methods in terms of the classification accuracy. [ABSTRACT FROM AUTHOR]

Negative selection algorithm (NSA) is an important kind of the one-class classification model, but it is limited in the big data era due to its low efficiency. In this paper, we propose a new NSA based on Voronoi diagrams: VorNSA. The scheme of the detector generation process is changed from the traditional “Random-Discard” model to the “Computing-Designated” model by VorNSA. Furthermore, we present an immune detection process of VorNSA under Map/Reduce framework (VorNSA/MR) to further reduce the time consumption on massive data in the testing stage. Theoretical analyses show that the time complexity of VorNSA decreases from the exponential level to the logarithmic level. Experiments are performed to compare the proposed technique with other NSAs and one-class classifiers. The results show that the time cost of the VorNSA is averagely decreased by 87.5% compared with traditional NSAs in UCI skin dataset. [ABSTRACT FROM AUTHOR]

With the advent of Big Data, data is being collected at an unprecedented fast pace, and it needs to be processed in a short time. To deal with data streams that flow continuously, classical batch learning algorithms cannot be applied and it is necessary to employ online approaches. Online learning consists of continuously revising and refining a model by incorporating new data as they arrive, and it allows important problems such as concept drift or management of extremely high-dimensional datasets to be solved. In this paper, we present a unified pipeline for online learning which covers online discretization, feature selection and classification. Three classical methods—the k -means discretizer, the χ 2 filter and a one-layer artificial neural network—have been reimplemented to be able to tackle online data, showing promising results on both synthetic and real datasets. [ABSTRACT FROM AUTHOR]

The major drawback of microarray data is the ‘curse of dimensionality problem’, this hinders the useful information of dataset and leads to computational instability. Therefore, selecting relevant genes is an imperative in microarray data analysis. Most of the existing schemes employ a two-phase processes: feature selection/extraction followed by classification. In this paper, a statistical test, ANOVA based on MapReduce is proposed to select the relevant features. After feature selection, MapReduce based K-Nearest Neighbor (K-NN) classifier is also proposed to classify the microarray data. These algorithms are successfully implemented on Hadoop framework and comparative analysis is done using various datasets. [ABSTRACT FROM AUTHOR]

Classification is a supervised learning task where a training set is used to construct a classifi- cation model, which is then used to predict the class of unforeseen test instances. It is often beneficial to use only a subset of the full training set to construct the classification model, and Instance Selection is the task of selecting the most appropriate subset of the training set. In many cases, the classification model induced from the reduced training set can have bet- ter predictive accuracy on test instances. ADR-Miner is a recently introduced Ant Colony Optimization algorithm for Instance Selection that aims to produce classification models with improved test set predictive accuracy. In this paper, we present an extension of ADR-Miner, where one classification algorithm is employed in the instance selection process, and potentially a different algorithm is employed in the final model construction phase. We evaluate perfor- mance using 37 UCI datasets, and we note the combinations of algorithms which produce the best results. [ABSTRACT FROM AUTHOR]

Abstract: As the need to analyze big data sets grows dramatically, the role that classification algorithms play in data mining techniques also increases. Big data analysis requires more of the data sets’ characteristics to be included, such as data structure, variety of sources, and the rate of update frequency. In this paper, we evaluate scenarios that examine which data set characteristics most affect the classification algorithms’ performance. It is still a complex issue to determine which algorithm is how strong or how weak in relation to which data set. Thus, our research experimentally examines how data set characteristics affect algorithm performance, both in terms of accuracy and in elapsed time. To do so, we use a multiple regression method to evaluate the causality between data set characteristics as independent variables, and performance metrics as dependent variables. We also examine the role that classification algorithms play as moderator in this causality. All benchmark data sets in a UCI database are used that are fit to run the classification algorithm. Based on the results of the experiment, we discuss the requirements of legacy classification algorithms to address big data analysis in a new business intelligence era. [Copyright &y& Elsevier]

The paper describes some methods of extracting knowledge from large amounts of data, and also the concepts of classification, regression and clustering. In terms of classification, it describes some of the techniques and methods, namely the decision trees, the Bayesian method, k- NN, etc. The steps of a process solving clustering and the K-means algorithm used are also described. [ABSTRACT FROM AUTHOR]

Despite more than two decades of progress, learning from imbalanced data is still considered as one of the contemporary challenges in machine learning. This has been further complicated by the advent of the big data era, where popular algorithms dedicated to alleviating the class skew impact are no longer feasible due to the volume of datasets. Additionally, most of existing algorithms focus on binary imbalanced problems, where majority and minority classes are well-defined. Multi-class imbalanced data poses further challenges as the relationship between classes is much more complex and simple decomposition into a number of binary problems leads to a significant loss of information. In this paper, we propose the first compound framework for dealing with multi-class big data problems, addressing at the same time the existence of multiple classes and high volumes of data. We propose to analyze the instance-level difficulties in each class, leading to understanding what causes learning difficulties. We embed this information in popular resampling algorithms which allows for informative balancing of multiple classes. We propose an efficient implementation of the discussed algorithm on Apache Spark, including a novel version of SMOTE that overcomes spatial limitations in distributed environments of its predecessor. Extensive experimental study shows that using instance-level information significantly improves learning from multi-class imbalanced big data. Our framework can be downloaded from https://github.com/fsleeman/minority-type-imbalanced. • First complete framework for learning from multi-class imbalanced big data. • Informative multi-class sampling methods that use instance-level characteristics. • Novel oversampling modification dedicated to MapReduce environments. • Code and data repository for reproducibility and applications of proposed methods. [ABSTRACT FROM AUTHOR]

Spam email automated analysis and classification are a challenging task, which is vital in the identification of botnet structures and cybercrime fighting. In this work, we propose an automated methodology and the resulting framework based on innovative categorical divisive clustering, used both for grouping and for classification of spam messages. In particular, the grouping is exploited to identify campaigns of similar spam emails, while the classification is used to label specific emails according to the goal of spammer (e.g., phishing, malware distribution, advertisement, etc.). This work introduces the CCTree algorithm, both as clustering algorithm and as classification algorithm, in two operative modes: batch and dynamic, to handle both large data sets and data streams. Afterward, the CCTree is applied to large sets of spam emails for campaign identification and labeling. The performance of the algorithm is reported for both clustering and classification, and a comparison between the batch and dynamic approaches is presented and discussed. [ABSTRACT FROM AUTHOR]

An imbalanced dataset is commonly found in at least one class, which are typically exceeded by the other ones. A machine learning algorithm (classifier) trained with an imbalanced dataset predicts the majority class (frequently occurring) more than the other minority classes (rarely occurring). Training with an imbalanced dataset poses challenges for classifiers; however, applying suitable techniques for reducing class imbalance issues can enhance classifiers' performance. In this study, we consider an imbalanced dataset from an educational context. Initially, we examine all shortcomings regarding the classification of an imbalanced dataset. Then, we apply data-level algorithms for class balancing and compare the performance of classifiers. The performance of the classifiers is measured using the underlying information in their confusion matrices, such as accuracy, precision, recall, and F measure. The results show that classification with an imbalanced dataset may produce high accuracy but low precision and recall for the minority class. The analysis confirms that undersampling and oversampling are effective for balancing datasets, but the latter dominates. [ABSTRACT FROM AUTHOR]

The identification of social media communities has recently been of major concern, since users participating in such communities can contribute to viral marketing campaigns. In this work, we focus on users’ communication considering personality as a key characteristic for identifying communicative networks i.e., networks with high information flows. We describe the Twitter Personality based Communicative Communities Extraction (T-PCCE) system that identifies the most communicative communities in a Twitter network graph considering users’ personality. We then expand existing approaches in users’ personality extraction by aggregating data that represent several aspects of user behavior using machine learning techniques. We use an existing modularity based community detection algorithm and we extend it by inserting a post-processing step that eliminates graph edges based on users’ personality. The effectiveness of our approach is demonstrated by sampling the Twitter graph and comparing the communication strength of the extracted communities with and without considering the personality factor. We define several metrics to count the strength of communication within each community. Our algorithmic framework and the subsequent implementation employ the cloud infrastructure and use the MapReduce Programming Environment. Our results show that the T-PCCE system creates the most communicative communities. [ABSTRACT FROM AUTHOR]

This review attempts to comprehend the insights and foresights into the grasping power of data science, data quality, data process, data pre-process, big data, big data process and analysis, analytics, BD (Big Data) and analytics lifecycle, file storage, platforms/technologies supported, Hadoop concepts, eco-system components, and design principles. Principle and philosophy behind computations are also explained through flow diagram. Various analytics based on its solutions, cluster computing based platforms like Apache Spark (its architecture -- core, other components, and utilities), MLlib package -- Machine Learning (ML) methods/ tasks and detailed supported algorithms are exclusively elucidated, to understand the concepts of these pinpoints. The explored comprehensive contents would definitely be useful and provide core understanding knowledge in the large scale ML dependent algorithms process, suitable to build the relevant application solutions (may be predictions/ classifications/ segmentations/ recommendations) via Apache -- Spark environment. [ABSTRACT FROM AUTHOR]

It is a common assumption that digital technology stores and retrieves text differently than physical libraries do. Websites and digitized text uploaded on them are retrieved via algorithms, tied together by links, and so on, whereas physical libraries are structured by classification schemes, catalogs, and indexes. Nevertheless, looking further back in history, it can be argued that digital technology in fact makes us store and retrieve text as humanity did to begin with—before the invention of the printing press. This article argues how. [ABSTRACT FROM AUTHOR]

Recent years huge amount of information are available in the World Wide web. 'Text Document Featured Clustering and Classification Using Fuzzy Logic' aims at obtaining effective clustering of text documents which can simply browsing of large volume of data set using semantic relationship between the words. Text clustering is mainly used for clustering a set of documents based on the user typed key term. 'A Fuzzy similarity-based algorithm' automatically classify a group of documents into set of groups with frequent concept. The set of documents and key terms which are entered by the user are pre-processed. The proposed system uses Fuzzy Featured Clustering(FFC) algorithm to identify the semantic relationship of words to create concepts. Text is classified based on features into number of clusters. The frequency of words are weighted by weighted matrix. The relationship between words like synonyms, hypernyms, hyponyms can also be identified. The proposed system is found more accurate, scalable and effective when compared to existing text clustering algorithms. [ABSTRACT FROM AUTHOR]