Search

To address the problem that traditional similarity-based outlier detection algorithms were not effective enough on high-dimensional unbalanced datasets, this paper proposed a novel Ensemble learning and Random projection-based Outlier Detection (EROD） framework. Firstly, the EROD algorithm integrated several random projection methods to reduce the dimensionality of high-dimensional data, which improved the data diversity. Secondly, it integrated several different traditional outlier detectors to build a heterogeneous ensemble model, which increased the robustness of the algorithm. Finally, the EROD acquired the final outlier value of the object by using the heterogeneous ensemble model to train the reduced dimensional data and by using two optimal combinations of the trained model to reduce the total error, and the algorithm determined the object with high outlier value as outlier point. The results showed that the algorithm had an average improvement of 3.6% and 14.45% in AUC and Precision @ n value compared with the traditional outlier detection algorithm and the outlier detection algorithm based on ensemble learning. Therefore, the EROD algorithm has the advantage of handling the anomalies of high-dimensional unbalanced data. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Outliers are a class of objects with different properties from other normal points, whose detection technology in various industries has a wide application to maintain the purity of data and ensure the safety of the industry. Most of the existing algorithms a re based on distance, density, and other traditional methods to detect outliers. This paper assigned each object an "isolation degree", the degree of isolation of the point relative to adjacent points, which could identify outliers by sorting, that was more efficient. It proposed the detection technology APO by improving and optimizing the AP clustering algorithm. It introduced the outlie r module and processed the isolated information of points. In addition, it added the amplification factor to make the difference between the outliers and the normal points more obvious. By increasing the sensitivity of the algorithm to outliers, it improved the accuracy of the algorithm. The experiment used simulated dataset real datasets, who ' s the results showed that the algorithm was more sensitive and it detected outliers more accurately than AP algorithm. In addition, this algorithm can cluster outliers while detecting outliers, which is not available in other detection algorithms. [ABSTRACT FROM AUTHOR]

As a new platform in big data stream computing, Apache Heron ignores the difference in communication modes between task instances and the unbalance of processing load among nodes, which leads to the decline system performance. To address the problem, this paper designed the model of node resource limitation, the model of communication overhead optimization and the model of data stream relationships among instances, as the foundation to propose the TUR-Heron. The strategy was composed of the node resource limitation algorithm and the instance reallocation algorithm. By judging the criteria for instance reallocation and executing instance reallocation algorithm, this strategy transformed the inter-node data streams into intra-node data streams and minimized the communication overhead of the system. The experimental results show that under the three sets of benchmarks, TUR-Heron reduces the communication overhead between nodes and the response latency of the system compared with the default scheduling strategy, and improves the balance of resource utilization of computing nodes. [ABSTRACT FROM AUTHOR]